Elon Musk's xAI deployed Colossus, a 100,000 H100 NVIDIA GPU supercomputer built and operational in just three months, with H200 upgrades planned.

Elon Musk's venture xAI brought its Colossus supercomputer online on Labor Day after completing development in 122 days from start to finish. According to Musk, Colossus is "the most powerful AI training system in the world," featuring 100,000 NVIDIA H100 data center GPUs, making it the largest training cluster to use such a huge number of H100s.

Musk announced that Colossus will double in size to 200,000 GPUs within a few months through the addition of 50,000 H200 GPUs, which is the flagship datacenter GPU using the Hopper architecture. The H200 is significantly more powerful than the H100, bringing almost 45% higher compute performance in specific generative AI and HPC applications.

The xAI Colossus project was started in June in Memphis with training commencing in July. The supercomputer is designed to prepare GROK 3 for delivery by December, replacing GROK 2. This new supercluster emerged after xAI ended its server rental deal with Oracle, with Colossus now delivering more power than what Oracle could provide and set to double in performance in a few months with the addition of 50,000 H200 GPUs.

The H200 brings almost 61GB higher memory and a significantly higher memory bandwidth of 4.8TB/s compared to 3.35TB/s on the H100. However, the H200 consumes 300W more power and requires liquid cooling, similar to the H100s already deployed in Colossus. Currently, Colossus is the only supercomputer that has reached 100,000 NVIDIA GPUs, followed by Google AI with 90,000 GPUs, OpenAI with 80,000 H100 GPUs, Meta AI with 70,000 GPUs, and Microsoft AI with 60,000 GPUs.