Liftr Insights data shows running NVIDIA AI instances costs approximately $2 million annually in power consumption.
Liftr Insights, a pioneer in market intelligence driven by unique data, conducted calculations for 2,000 NVIDIA cards to demonstrate the ongoing costs for running popular AI workloads. NVIDIA has been dominating news and the markets for its semiconductors, with their latest models such as the Hopper H100 considered essential to the rising demand for AI training and other artificial intelligence processes. Liftr data show that even earlier types like the A100 remain in high demand.
For a $33M investment in AI accelerator components and $2M in electric consumption per year, a company could be running 1,000 H100s and 1,000 A100s in Dallas, Texas, which combined could be generating high performance in excess of 44.7 FP64 petaflops. However, the same infrastructure would incur different operating costs depending on location. Running this infrastructure in Houston would generate a larger power bill at approximately $2.1M per year, while annual costs would be less in San Antonio or Austin, at $1.9M and $1.6M, respectively.
"Despite the news of the delay in the Blackwell processes," says Tab Schadt, CEO of Liftr Insights, "major cloud providers like AWS, Azure, and GCP have been increasing their adoption of the latest NVIDIA semiconductors." Liftr Insights, which tracks the top 6 cloud providers and 3 trending providers, shows the adoption growth of the Hopper brand, with the H100 being the most widely discussed model within the Hopper series.
In addition to showing common configurations as well as adoption trends by major consumers, Liftr data can provide deeper insight for data center operators. "We help our customers understand the impact of these new chips on output and their financials," says Schadt. "When looking at new AI, it's more than knowing what's available. Rather, for specific configurations, it's understanding the performance, power consumption, and ultimately, the bottom line of on-going costs."