The hybrid cooling lie: 30% of your server is still running hot

Key Takeaways:

• Direct-to-chip cooling caps out at 60–80% of a server's heat. The remaining ~30% still depends on air, which moves far less heat per unit volume than liquid.
• Heat scales with rack density and leads to side effects like DIMM thermal throttling and louder, more power-hungry fans
• Removing air dependency with precision liquid cooling eliminates fan power and noise and frees up density headroom

AI and high-performance computing (HPC) are changing the data center cooling equation. As processors get more powerful and rack densities rise, data centers generate far more heat in a much smaller footprint. At those densities, air cooling alone is just not good enough.

That is why the industry has increasingly turned to direct-to-chip or cold plate cooling. The approach makes sense. Put a cold plate directly on the CPUs and GPUs generating the most heat and use liquid to carry that heat away.

But there is a catch. The rest of the server (memory, networking and power supply) still relies on air cooling.

This is what we call the hybrid cooling lie: the assumption that applying liquid cooling to the hottest components solves the entire cooling challenge. In reality, direct-to-chip systems still leave about 30% of a server's heat for air to cool. At lower rack densities, that may be manageable. As densities climb, that incremental heat becomes much harder to ignore.

Where the other 30% comes from

A typical direct-to-chip liquid cooling system removes roughly 60% to 80% of a server's heat by cooling the CPUs and GPUs directly. That leaves about 30% to be cooled by traditional air cooling. The remaining heat comes from components that play an important role in the server’s functionality. Failing to cool memory modules, voltage regulators, network interface cards, storage and power supplies can create real performance and reliability problems for the entire server.

DIMMs are particularly sensitive to temperature and sit close to the processor sockets. Voltage regulator modules (VRMs) handle the large electrical currents required by today’s AI chips. High-speed network interface cards (NICs) support the low-latency communication AI clusters depend on and generate more heat as throughput increases. Storage devices and power supply units (PSUs) play a part in the server's overall heat load.

In a hybrid system, air still must cool all of them. Air is limited in its ability to cool critical components beyond GPUs and CPUs; it’s far less effective at carrying heat than liquid, moving thousands of times less heat per unit volume. As a result, air remains the weak link in the server cooling system.

The math changes at higher densities

At 10 kW per rack, leaving 30% of the heat to air cooling means managing about 3 kW through server fans and room-level computer room air handlers (CRAHs). Sounds relatively easy to handle, right?

But take that same 30% and put it in a 100 kW rack. The air-cooling system is now being asked to remove 30 kW of heat. That’s more heat than an entire traditional air-cooled server rack once generated, and it’s creating a few problems.

The first is thermal throttling. Direct-to-chip cooling might keep the CPU at a manageable temperature, but nearby memory still relies on air that has already picked up heat from other components. Independent testing in controlled server wind tunnel facilities found that memory modules in hybrid systems with fixed fan speeds can reach their thermal limits. This aligns with DDR4 and DDR5 specifications, where the memory controller can reduce frequency or voltage to prevent damage.

Then there are the fans. The harder the air has to work to cool the remaining components, the faster those fans need to run. That means more noise and more electricity spent moving air. Some of the efficiency gained by liquid-cooling the CPUs and GPUs is then lost to keeping everything else cool.  

There is also a facility-level impact. High-performance memory still needs relatively cool inlet air to operate effectively. That can limit how much operators raise data center temperatures and their ability to take advantage of efficient, water-free dry cooling. The CPU may be perfectly comfortable, but the memory, NICs and PSUs can still overheat. That leaves operators running chillers and evaporative cooling to protect the portion of the server that still depends on air.

What happens when you cool the whole server?

Another way to approach the problem is to stop treating the CPU and GPU as the entire cooling challenge and start treating the server as one thermal system. Full-chassis liquid immersion, also known as chassis-level precision liquid cooling, brings liquid cooling to all the heat-generating components inside the server.

This is different from traditional open-bath immersion, where entire servers are submerged in large tanks of dielectric fluid. Those systems can require significant changes to how equipment is installed, accessed and maintained.

With chassis-level precision liquid cooling, the server stays in a familiar rack-mounted form factor. Each server sits inside its own sealed chassis, with a relatively small amount of dielectric fluid circulating inside. The fluid first flows to the highest-heat components, including the CPUs and GPUs, before moving across the rest of the system. Memory, VRMs, storage, PSUs and high-speed NICs are all cooled by the same nonconductive fluid. Instead of capturing 70% of the server's heat with liquid and asking air to handle the rest, the goal is to capture 100% of the heat into the liquid loop.

What changes when air is removed from the equation?

Once the server no longer depends on air cooling, the benefits go beyond keeping the CPU cool.

First, removing fans eliminates a source of power consumption and dramatically reduces noise. If you spend less power moving air, more of that power can go to compute. Fans are also prone to failure. Removing them reduces maintenance costs. It also changes what is possible with density. Airflow requires space. Removing that requirement gives system designers more flexibility to increase compute density within the same physical footprint.

Then there is the temperature of the cooling loop itself. If components don't need cool air, the liquid loop can operate at higher secondary inlet temperatures. Research and deployments at U.S. Department of Energy national laboratories have already shown that warmer water can effectively cool high-performance systems while reducing the need for energy-intensive chillers.

Higher operating temperatures also create another opportunity: rejecting heat without relying on evaporative cooling. If the captured heat is warm enough relative to the outdoor air, closed-loop dry coolers can reject that heat outside the building without evaporating water. That can greatly reduce or eliminate cooling-related water use.

Beyond the hybrid compromise

Direct-to-chip cooling has played an important role in the industry's transition away from traditional air cooling. It addresses the hottest components first and gives operators a practical way to introduce liquid cooling into existing environments.

But AI infrastructure is not standing still. It’s time to consider a new approach.