Data processing units (DPUs), intelligent processing units (IPUs), and SmartNICs reduce data center power consumption through hardware offloading.
Server CPUs excel at executing diverse workloads, but they are not the optimal answer to every computing problem. CPUs trade off universality for performance and efficiency, whereas custom hardware delivers better performance with lower power consumption if the task at hand is well defined or encapsulated. Many tasks currently executed by CPU-based servers in the data center, including IPsec and Open vSwitch (OVS), represent opportunities for this kind of specialization. This is likely to change for next-generation servers if AMD's, Broadcom's, Intel's, Marvell's, and Nvidia's plans for infrastructure offload engines—referred to as SmartNICs, Infrastructure Processing Units (IPUs), and Data Processing Units (DPUs)—are realized.
Data centers face a genuine power crisis, having maxed out on power consumption as a result of both the electrical power that servers consume and the infrastructure required to power and cool these servers. Data center service expansion currently relies on either becoming more efficient within the existing power and cooling footprint or building another data center. Cloud service providers (CSPs), communications service providers (CoSPs), and enterprise data centers have pursued both strategies, with CPU vendors adding more processor cores on increasingly smaller semiconductor process nodes while companies continue to build new facilities.
The semiconductor industry has been offering programmable SmartNICs capable of offloading infrastructure tasks for several years. AMD/Xilinx offers FPGA-based Alveo adaptable accelerator cards; Intel has migrated from SmartNICs to FPGA- and ASIC-based IPUs to offload infrastructure tasks; and Nvidia, which purchased Mellanox in 2020, offers a line of Bluefield DPUs for "offloading, accelerating, and isolating a broad range of advanced networking, storage, and security services." These cards are designed as PCIe boards that plug into data center servers to accelerate infrastructure tasks while reducing power consumption.
Nvidia published a white paper titled "DPU Power Efficiency" that quantifies the benefits of these accelerators: "One of the best ways to improve efficiency is to use a Data Processing Unit (DPU) or SmartNIC to offload and accelerate networking, security, storage or other infrastructure functions and control-plane applications, which reduces server power consumption up to 30%. The amount of power savings increases as server load increases and can easily save $5.0 million in electricity costs for a large data center with 10,000 servers over the 3-year lifespan of the servers, plus additional savings in cooling, power delivery, rack space, and server capital costs." The paper's technical examples, based on trials by Ericsson, Nvidia, and other companies, demonstrate the power savings and return on investment achievable with infrastructure offload engines.
The competitive question is not whether these accelerators will see use—the economics are undeniable—but which company's vision will prevail. Nvidia expects that next-generation servers installed in the next round of upgrades will include infrastructure accelerator cards pre-integrated. VMware's Project Monterey and Red Hat's OpenShift Container Platform already incorporate the means to offload infrastructure tasks to supported accelerators, and the Linux Foundation has launched the Open Programmable Infrastructure (OPI) project. TIRIAS Research believes infrastructure acceleration will appear starting with the next wave of server upgrades. Intel, meanwhile, has broader goals: during Architecture Day 2021, Intel's Guido Appenzeller, then CTO of the Intel Data Platforms Group, outlined a vision moving IPUs to the core of the data center, where they would manage the construction and teardown of composable systems using CPUs, memory, and storage as interchangeable parts in dynamic assembly and disassembly processes.