NVIDIA invests $20 billion in R&D for next-generation AI chip technology beyond current GPU architectures.
On the day before Christmas, Nvidia announced a $20 billion transaction to license technology from chip startup Groq and hire key employees, including its CEO Jonathan Ross, who previously helped Google create the Tensor Processing Units (TPUs) that have become the leading alternative to Nvidia's AI processors. This significant competitive move is set to receive substantial attention at Nvidia's annual GPU Technology Conference (GTC) in San Jose, California, taking place at the San Jose McEnery Convention Center, with CEO Jensen Huang's keynote address held at the nearby SAP Center. On Nvidia's late February earnings call, Jensen stated, "I've got some great ideas that I'd like to share with you at GTC," signaling plans to share the company's vision for incorporating Groq's chip technology into its AI computing ecosystem. The conference, which has been dubbed the "Super Bowl of AI," will also include updates on Nvidia's roadmap for its next-generation Vera Rubin family of graphics processing units (GPUs).
Nvidia is expected to use Groq's technology to build a brand-new chip targeting inference—the daily use of AI models—a growing and increasingly competitive market segment. While Nvidia's GPUs dominate the training stage of AI computing, where models are prepared for real-world usage, inference has become more crowded as AI adoption goes mainstream and customers seek cost-effective solutions. Despite this competition, Nvidia disclosed that approximately 40% of its revenue in 2024 came from inference, and finance chief Colette Kress recently highlighted that industry publication SemiAnalysis "declared Nvidia inference king," noting that its current generation Grace Blackwell GPUs offer massive performance improvements over its predecessor Hopper. At last year's GTC, Jensen told analysts that "the vast majority of the world's inference is on Nvidia today."
The inference market, however, features several formidable competitors. Advanced Micro Devices, the distant No. 2 maker of GPUs, recently signed up Meta Platforms as a customer in a major partnership announcement. Google's in-house Tensor Processing Units, which Google co-designs with Broadcom, are formidable challengers in both training and inference, and Google's Tensor Processing Units-based Gemini chatbot has elevated their reputation as Nvidia's biggest threat. Amazon has touted its in-house Trainium chip's capabilities in both tasks, and Anthropic, the AI startup behind the Claude model, uses Trainium alongside TPUs and a deal with Nvidia inked in the fall. Additionally, Cerebras, an AI startup preparing for an initial public offering, has garnered attention, with Oracle co-CEO Clay Magouyrk recently name-dropping the company on its earnings call.
Nvidia evidently saw an opportunity to strengthen its inference offerings, which prompted the $20 billion investment in Groq. Rather than acquiring Groq entirely—which could have triggered antitrust scrutiny—the deal was structured as a non-exclusive licensing agreement, allowing Groq to continue operating its inference cloud service running on its specialized chips. Jonathan Ross, Groq's founder and now-former CEO, moved to Nvidia as chief software architect; before founding Groq in 2016, Ross was part of the Google team that developed the original TPU. Groq had developed what it called an inference-focused LPU, short for Language Processing Units. Ross has consistently stated in podcast interviews that Groq didn't attempt to compete with Nvidia on training; instead, the startup identified inference computing as the area where it could innovate and establish its position.