Every computation of a GPU consumes power. All that power ends up as heat. Although it’s produced as a result of the computation, strictly speaking, the minimum thermodynamic cost of the computation itself - the Landauer bound1,2 - is many orders of magnitude below the actual consumption, so effectively all electrical power is dissipated as heat.
The computations occur, in terms of power consumed, almost all at the GPU and CPU level, but also in other components such as memory drives, PCIe, networking etc. This means that the GPUs and CPUs, generate power internally (an internal power source). At steady state, the amount of power generated is equal to the amount dissipated meaning that the temperature of the components themselves are constant (we are as said, at the equilibrium steady-state).
Nevertheless, that temperature must be kept under a certain threshold, for performance and physical integrity. This is the reason why cooling is required: to keep the temperature of the heat generating components under a certain threshold. If each component couldn’t dissipate that power, they would otherwise overheat, and reach temperatures beyond their integrity threshold, thus resulting in failure.
Let’s study the physics, with a simplified but still very useful example. In the limit case, a homogeneous thermal mass (an object considered all equal in properties - this makes the math easy), also called a heat sink/source, that exchanges heat isotropically (in all directions equally) that is subjected to internal heat generation, is governed (i.e. the behavior depends upon) an equation (an equation is the mathematical representation of a physical phenomenon - heat transfer) like:
The equation is a time equation, it tells us how the temperature of the thermal mass (heat sink) T changes in time (t). If we extract more than we produce (Q̇diss > Q̇gen), the temperature reduces (we are cooling the object), if we extract less (Q̇diss < Q̇gen), the object is heating. If we extract the same as we generate, the temperature stays the same (the steady-state mentioned above). This is what we want to do: ride at a temperature that guarantees our performance and that is under that integrity threshold. Knowing the energy consumption of the GPUs and CPUs, and knowing - as explained above - that all the (electrical) power becomes heat, this number (the heat generation) determines the whole heat transfer problem. This also teaches us another thing: heat is energy (thermal energy) that moves from one object (this case the homogeneous thermal mass) to another one (in this case the environment outside).
So, we understood that heat needs to be extracted from the chips (CPUs or GPUs) and dumped somewhere (energy can’t be destroyed right? so we can only dump it!). Well, that “somewhere” is always the ambient (the air outside or a body of water). Now, how we get that heat there is the engineering: essentially, we will pass heat through multiple mediums until it gets there, in possibly cheap, lower power, efficient, sustainable ways.
How do we do this? with a heat exchanger
A heat exchanger is a heat transfer device that is used for transfer of internal thermal energy between two or more fluids available at different temperatures. In most heat exchangers, the fluids are separated by a heat transfer surface, and ideally they do not mix.
Heat exchangers appear in almost every industry, process, power, petroleum, transportation, air-conditioning, refrigeration, cryogenic, heat recovery, alternate fuels and more. HE is typically done without the two liquids (typically liquids or gases, mixing) - this is called indirect contact HE. The heat transfer thus between two separate liquids, through a surface (or core) in which heat is transferred via conduction.
In heat transfer, the ability to move heat from one phase to another, depends proportionally to surface area: to move heat from say one liquid to another, an example is using two concentric tubes, one internal (the internal flow) and one external (the annulus flow). Adding surface areas, typically as objects like fins, on the exterior of the internal wall improves the heat transfer.
On a chip, the first part of our heat exchanger is a series of fins (in firm contact with the power generating chip as a heat sink, see pictures below) - i.e. the core/matrix the principal part of the first heat exchanger! - which transfer the heat to forced air (pushed by a fan, air (option 1)) or to liquid (option 2 - pushed by a pump). When we use a liquid, we try to avoid consuming it, so the conventional solution is to run closed loop, i.e. the same liquid circulated in a closed system (in a perfectly designed system you never need to top up - in practice real use accounts to 5% loss everywhere) in which the water is heated by the chip, moved along a pipe, cooled by a HE and then resent, cold, back to the chip.


The heated air or liquid, then needs to dump its heat to the environment. Excluding evaporative cooling (you transfer the heat to the environment by evaporating water in a cooling tower - which consumes it!) as industry is moving away, this can be done by further moving the heated air outside, via a series of fans, or by extracting the heat from the liquid to the ambient via another heat exchanger (an air-cooled heat exchanger - ACHE). This one typically involves the liquid in running in a pipe with external fins immersed in air in cross flow (the direction of air is orthogonal to the direction of the liquid). Often this is also called dry cooler.

Now, the reason why the headlines speak about liquid cooling (beyond speaking about how bad open loop cooling is) are the following. Water by nature has a high specific heat capacity (property of a material of absorbing heat for a given increase in temperature) and has a high heat transfer coefficient (property of water and the core of exchanging heat). Air performs poorly on both. To achieve the same performance of water, you would need much more mass flow of air (more mass to counteract the lower specific heat capacity) and better surface conditions (to counter balance the poor heat transfer coefficient). This means liquid is a very compact cooling solution to extract heat from chips, compared to air-cooled systems. Also, running the numbers, it takes less power of the liquid pumps to extract the same heat from the chip compared to the air fans.
| Property | Water | Air | Implication |
|---|---|---|---|
| Specific heat capacity (cₚ) | ~4,180 J/(kg·K) | ~1,005 J/(kg·K) | Water absorbs ~4× more heat per kg per degree of temperature rise |
| Density (ρ) | ~997 kg/m³ | ~1.2 kg/m³ | Per unit volume, water carries ~3,500× more heat (ρ·cₚ: ~4.17 MJ/m³·K vs ~1.2 kJ/m³·K) |
| Volumetric heat capacity (ρ·cₚ) | ~4.17 MJ/(m³·K) | ~0.0012 MJ/(m³·K) | For the same heat load and ΔT, air needs ~3,500× the volumetric flow rate |
| Convective heat transfer coefficient (h) | ~1,000–20,000 W/(m²·K) (forced, single-phase) | ~10–150 W/(m²·K) (forced) | Water exchanges heat with the cold plate ~100× more effectively per unit area |
So, to cool or not to cool? Cool. To air cool or liquid cool? It depends. Air cooling systems are in principle easier and less complex. However, they also occupy more space and can be louder than the liquid cool system. Moreover, they also require advanced filtration systems upstream of the racks, compared to liquid-cooled systems in which the server is effectively sealed from the outside environment.
Sources
- R. Landauer, “Irreversibility and Heat Generation in the Computing Process,” IBM Journal of Research and Development 5(3) (1961) 183–191 — establishes the minimum thermodynamic cost of computation, kT·ln 2 per bit erased.
- A. Bérut et al., “Experimental verification of Landauer’s principle linking information and thermodynamics,” Nature 484 (2012) 187–189 — the experimental confirmation.
- Ebrahimi, Jones & Fleischer, “A review of data center cooling technology, operating conditions and the corresponding low-grade waste heat recovery opportunities,” Renewable and Sustainable Energy Reviews 31 (2014) 622–638 — states that essentially all electrical energy delivered to servers is converted to heat, and builds the entire heat-recovery case on it.
- Thulukkanam, Kuppan. Heat Exchanger Design Handbook, Vol. 2. Boca Raton, FL: CRC Press, 2013.
At Ambra, we build and operate GPU nodes at renewable generation sites — which means the path from the chip to the ambient is something we design, not something we inherit.
← All posts