AWS is deploying custom IRHX cooling solutions for Nvidia GPU clusters instead of relying on third-party cooling vendors.

Amazon Web Services has developed a proprietary cooling system called the In-Row Heat Exchanger, or IRHX, to handle the thermal demands of Nvidia's latest GPU architectures. According to Dave Brown, VP Compute and ML Services at AWS, existing liquid cooling solutions were inadequate for the company's needs. "They would take up too much data center floor space, would still require major modifications to data centers, or increase water usage substantially," Brown said in a presentation. He added that "while some of these solutions could work for lower volumes at other providers, they simply wouldn't be enough liquid cooling capacity to support our scale."

The IRHX system consists of a pumping unit, a water distribution cabinet, and fan coils that work together to cool AWS's hardware. Liquid cools the chips through a cold plate co-designed by AWS and Nvidia, then cycles back through the IRHX, where it is cooled and released. This design allows AWS to maintain thermal management without requiring redesigns of existing data center infrastructure.

The system powers AWS's most powerful EC2 instance, the P6e UltraServer, which includes the Nvidia GB200 NVL72. According to Brown, this configuration "enables 72 Nvidia Blackwell GPUs to act as a single massive GPU." The IRHX fits into existing AWS rack dimensions and infrastructure, making it deployable across global data centers without modification to current facilities. As Brown explained, "With the IRHX we don't need to design the data center around the rack."

While currently paired with Nvidia's Blackwell-based systems, the IRHX is likely to be used with Amazon's own Graviton chips should their cooling requirements increase in the future. This custom cooling solution extends AWS's established strategy of building proprietary hardware—including chips and networking systems—to support its infrastructure at scale for AI workloads demanding both performance and computational power.