Nvidia announces GH200 Grace Hopper superchip integrating CPU and GPU with 141GB HBM3e memory for AI workloads.

Nvidia on Tuesday unveiled a revamped version of its next-generation Grace Hopper Superchip platform featuring HBM3e memory for artificial intelligence and high-performance computing. The new version of the GH200 Grace Hopper maintains the same Grace CPU and GH100 Hopper compute GPU but incorporates HBM3e memory that offers higher capacity and bandwidth compared to its predecessor.

The new GH200 Grace Hopper Superchip is built around a 72-core Grace CPU equipped with 480 GB of ECC LPDDR5X memory, paired with a GH100 compute GPU featuring 141 GB of HBM3e memory. The HBM3e is organized in six 24 GB stacks and uses a 6,144-bit memory interface, though Nvidia physically installs 144 GB with only 141 GB accessible for better yields.

The performance improvements over Nvidia's current GH200 Grace Hopper Superchip—which comes with 96 GB of HBM3 memory providing bandwidth of less than 4 TB/s—are substantial. The new model increases memory capacity by approximately 50 percent and raises bandwidth by over 25 percent. These enhancements enable the new platform to run larger AI models than the original version and deliver tangible performance improvements, particularly important for training workloads.

Regarding availability, Nvidia's GH200 Grace Hopper platform with HBM3 is currently in production and will be available commercially starting next month. The GH200 Grace Hopper platform with HBM3e is now sampling and is expected to be available in the second quarter of 2024. Nvidia stressed that the new GH200 Grace Hopper uses the same Grace CPU and GH100 GPU silicon as the original version, eliminating the need for new revisions or steppings. Both versions will co-exist on the market, with the HBM3e variant sold at a premium given its higher performance.

The new platform is fully compatible with Nvidia's MGX server specification, making it a drop-in replacement for existing server designs. Jensen Huang, chief executive of Nvidia, stated: "To meet surging demand for generative AI, data centers require accelerated computing platforms with specialized needs. The new GH200 Grace Hopper Superchip platform delivers this with exceptional memory technology and bandwidth to improve throughput, the ability to connect GPUs to aggregate performance without compromise, and a server design that can be easily deployed across the entire data center."