AMD launched the MI350 flagship GPU for AI datacenters, featuring 185 billion transistors for high-performance inference and training.
Advanced Micro Devices Inc. today introduced a new line of artificial intelligence chips called the Instinct MI350 series, which the company says can outperform Nvidia Corp.'s Blackwell B200 at some tasks. The product family includes two graphics cards: the top-end MI355X, which relies on liquid cooling to dissipate heat, and a scaled-down chip called the Instinct MI350X that trades off some performance for lower operating temperatures and can use fans instead of liquid cooling. According to Vamsi Boppana, the senior vice president of AMD's Artificial Intelligence Group, "With flexible air-cooled and direct liquid-cooled configurations, the Instinct MI350 Series is optimized for seamless deployment, supporting up to 64 GPUs in an air-cooled rack and up to 128 in a direct liquid-cooled and scaling up to 2.6 exaFLOPS of FP4 performance."
The MI350 series is based on a three-dimensional, 10-chiplet design where eight chiplets contain compute circuits made using Taiwan Semiconductor Manufacturing Co.'s latest three-nanometer process, sitting atop two six-nanometer I/O chiplets that function as the MI350's base layer and manage data flow inside the processor. Both the MI355X and the MI350X ship with 288 gigabytes of HBM3E memory, a variety of fast, high-capacity RAM widely used in AI chips. HBM3E theoretically supports up to 16 vertically layered RAM layers, and Micron Technology Inc.'s latest HBM3E chips ship with a Memory Built-In Self-Test module that reduces the amount of specialized equipment needed to develop AI chips including HBM3E memory.
According to AMD, the MI350 series features 60% more memory than Nvidia's flagship Blackwell B200 graphics cards. The company promises faster performance for some workloads, with AMD saying that MI350 chips can process 8-bit floating point numbers 10% faster than the B200 and 4-bit floating point numbers more than twice as fast. The MI350's support for four-bit floating point, or FP4, data is one of the improvements it introduces over earlier AMD graphics cards, allowing a single chip from the MI350 series to run an AI model with up to 520 billion parameters. AMD is also promising a 40% increase in tokens per dollar compared to competing products.
AMD will make the MI350 available in 8-chip server configurations that will provide up to 160 petaflops of performance for some FP4 workloads, with one petaflop corresponding to 1,000 trillion computations per second. Further down the line, AMD plans to launch a line of rack systems called Helios that will combine chips from the upcoming Instinct MI400 chip series, the successor to the MI350, with the company's central processing units and Pensando data processing units, which offload infrastructure management tasks from an AI cluster's other chips.
On the software side, Helios will ship with the company's ROCm platform, a collection of developer tools, application programming interfaces and other components used to program AMD graphics cards. ROCm 7.0, the latest release, enables AI models to perform inference more than 3.5 times faster than before and can triple the performance of training workloads. According to AMD, the speedup is partly the result of optimizations that allow ROCm 7.0 to manage data movement more efficiently and better handle distributed inference, the task of spreading an inference workload across multiple graphics cards to accelerate processing. Boppana wrote that "Over the past year, ROCm has rapidly matured, delivering leadership inference performance, expanding training capabilities, and deepening its integration with the open-source community."