Elon Musk's xAI announced plans to build Gigafactory of Compute by fall 2025 using 100,000 Nvidia H100 GPUs.

xAI, Elon Musk's AI startup, plans to build a massive supercomputer referred to as the Gigafactory of Compute to enhance its AI chatbot, Grok, according to Reuters citing The Information. The supercomputer is projected to be ready by fall 2025 and might involve a collaboration with Oracle. With this development, Musk aims to significantly surpass rival GPU clusters in both size and capability.

In a presentation for investors, Elon Musk revealed that the new supercomputer will use as many as 100,000 Nvidia's H100 GPUs based on the Hopper architecture, making it at least four times larger than the largest existing GPU clusters. Nvidia's H100 GPUs are highly sought after in the AI data center chip market, although strong demand made them difficult to obtain previously. However, these are no longer the range-topping Nvidia GPUs, with the company about to ship its H200 compute GPUs for AI and HPC applications and preparing to ship its Blackwell-based B100 and B200 GPUs in the second half of the year.

It is unclear why xAI decided to use essentially a previous-generation technology for its 2025 supercomputer, but this substantial hardware investment reflects the scale of xAI's ambitions. The plans could change, as this is an unofficial report, though Musk reportedly holds himself 'personally responsible for delivering the supercomputer on time' as the project is very important for developing large language models.

xAI seeks to compete directly with AI giants like OpenAI and Google. Musk, who also co-founded OpenAI, positions xAI as a formidable challenger in the AI space. The Grok 2 model required around 20,000 Nvidia H100 GPUs for training, and future iterations, such as Grok 3, will need as many as 100,000 GPUs, according to Musk.

Neither xAI nor Oracle provided comments on the collaboration when approached. Nonetheless, Musk's presentation to investors underscores his commitment to pushing the boundaries of AI technology through substantial infrastructure investments and strategic partnerships.