AMD acquires AI inference chip startup Taalas, hardwiring AI models into silicon for up to 17,000 tokens/second throughput.

AMD has acquired Toronto-based AI chip startup Taalas, marking its latest move to challenge Nvidia's dominance in AI hardware. The company has developed a radically different approach to inference by etching model weights directly into silicon, creating model-specific integrated circuits that promise to boost performance by an order of magnitude or more.

The deal, announced at market close on Thursday, parallels Nvidia's $20 billion licensing arrangement with Groq from December, positioning both acquisitions as bids to deliver faster and cheaper inference for high-performance AI services like code assistants and AI agents. While AMD did not disclose financial terms, the transaction represents a full acquisition rather than an acquihire.

Unlike conventional GPUs or dataflow architectures used by competitors like Groq and Cerebras, Taalas chips do not rely on HBM to store model weights. Instead, the weights are etched directly into silicon, making each chip essentially purpose-built for a specific model.

The technology is already proven. In February, Taalas unveiled its first test chip, the HC1, fabricated on TSMC's 6nm process. When serving Meta's Llama 3.1 8B model, the chip achieved 16,960 tokens per second—48 times faster than Nvidia GPUs and 8.5 times faster than Cerebras accelerators at the time of announcement.

While Llama 3.1 is now dated, having launched in mid-2024, the reticle-sized chip was designed primarily to validate the approach. Taalas has remained secretive about implementation details, but the chips comprise two main components: a mask-ROM recall fabric where weights are etched, and an SRAM recall fabric for storing KV caches and fine-tuning adapters.

The upcoming second-generation HC2 chip, due this summer, will support up to 20 billion parameters. Though modest for individual chips, this design scales efficiently through pipeline parallelism—50 accelerators would suffice to serve trillion-parameter models, leveraging AMD's existing rack-scale compute platforms and system design expertise.

This efficiency advantage is significant. Nvidia's recently announced LPX systems would require several dozen GPUs and at least 2,000 Groq LPUs to serve equivalent models. AMD plans to pair its Instinct-based Helios racks with Taalas-based accelerators in a disaggregated architecture, where GPUs handle computationally heavy prompt processing while token generation offloads to Taalas chips.

There is potential for a tick-tock cadence where customers initially deploy and validate models on Instinct accelerators before transitioning to Taalas accelerators once validated. AMD SVP of AI Vamsi Boppana stated: "AMD is building a full-stack AI platform that gives customers the flexibility to deploy the right compute solutions for every AI workload."

However, the technology carries a significant trade-off. Once deployed, the chips are locked to their specific model. Any change beyond LoRA adapter-scale modifications requires a full re-spin—expensive and time-consuming. With new frontier models arriving nearly monthly, customers will need high confidence in their model choices before committing to silicon.

That said, Taalas indicates the situation is manageable. Model changes do not require starting from scratch; only two metal layers need modification, substantially reducing cost and turnaround time. The company has stated that etching model weights into silicon costs roughly 100 times less than training a frontier model from scratch.

AMD is well-positioned to monetize this advantage. OpenAI, Anthropic, and Meta are all major Instinct customers, and close working relationships between model developers and the chip designer suggest that GPT or Claude deployments on combined Taalas and Instinct accelerators are plausible.

The technology also carries implications for model development. Test-time scaling—allowing models to reason longer before responding—can reduce hallucinations but requires substantially more tokens, increasing costs and latency. If Taalas can reduce per-token costs and boost output speeds by 10 to 20 times, developers may extend reasoning time even further, with immediate benefits for code assistants, chatbots, and agents.

Subject to regulatory approval, the acquisition is expected to close in the fourth quarter.