Microsoft debuts custom chips to boost data center security and power efficiency 

At the Ignite developer conference today, Microsoft unveiled two new chips designed for its data center infrastructure: the Azure Integrated HSM and the Azure Boost DPU. 

Scheduled for release in the coming months, these custom-designed chips aim to address security and efficiency gaps faced in existing data centers, further optimizing their servers for large-scale AI workloads. The announcement follows the launch of Microsoft’s Maia AI accelerators and Cobalt CPUs, marking another major step in the company’s comprehensive strategy to rethink and optimize every layer of its stack— from silicon to software—to support advanced AI.

The Satya Nadella-led company also detailed new approaches aimed at managing power usage and heat emissions of data centers, as many continue to raise alarms over the environmental impact of data centers running AI.

Just recently, Goldman Sachs published research estimating that advanced AI workloads are poised to drive a 160% increase in data center power demand by 2030, with these facilities consuming 3-4% of global power by the end of the decade.

The new chips

While continuing to use industry-leading hardware from companies like Nvidia and AMD, Microsoft has been pushing the bar with its custom chips.

Last year at Ignite, the company made headlines with Azure Maia AI accelerator, optimized for artificial intelligence tasks and generative AI, as well as Azure Cobalt CPU, an Arm-based processor tailored to run general-purpose compute workloads on the Microsoft Cloud.

Now, as the next step in this journey, it has expanded its custom silicon portfolio with a specific focus on security and efficiency. 

The new in-house security chip, Azure Integrated HSM, comes with a dedicated hardware security module, designed to meet FIPS 140-3 Level 3 security standards.

According to Omar Khan, the vice president for Azure Infrastructure marketing, the module essentially hardens key management to make sure encryption and signing keys stay secure within the bounds of the chip, without compromising performance or increasing latency.

To achieve this, Azure Integrated HSM leverages specialized hardware cryptographic accelerators that enable secure, high-performance cryptographic operations directly within the chip’s physically isolated environment. Unlike traditional HSM architectures that require network round-trips or key extraction, the chip performs encryption, decryption, signing, and verification operations entirely within its dedicated hardware boundary.

While Integrated HSM paves the way for enhanced data protection, Azure Boost DPU (data processing unit) optimizes data centers for highly multiplexed data streams corresponding to millions of network connections, with a focus on power efficiency. 

The offering, first in the category from Microsoft, complements CPUs and GPUs by absorbing multiple components of a traditional server into a single piece of silicon — right from high-speed Ethernet and PCIe interfaces to network and storage engines, data accelerators and security features.

It works with a sophisticated hardware-software co-design, where a custom, lightweight data-flow operating system enables higher performance, lower power consumption and enhanced efficiency compared to traditional implementations.

Microsoft expects the chip will easily run cloud storage workloads at three times less power and four times the performance compared to existing CPU-based servers.

New approaches to cooling, power optimization

In addition to the new chips, Microsoft also shared advancements made towards improving data center cooling and optimizing their power consumption.

For cooling, the company announced an advanced version of its heat exchanger unit – a liquid cooling ‘sidekick’ rack. It did not share the specific gains promised by the tech but noted that it can be retrofitted into Azure data centers to manage heat emissions from large-scale AI systems using AI accelerators and power-hungry GPUs such as those from Nvidia.

On the energy management front, the company said it has collaborated with Meta on a new disaggregated power rack, aimed at enhancing flexibility and scalability.

“Each disaggregated power rack will feature 400-volt DC power that enables up to 35% more AI accelerators in each server rack, enabling dynamic power adjustments to meet the different demands of AI workloads,” Khan wrote in the blog.

Microsoft is open-sourcing the cooling and power rack specifications for the industry through the Open Compute Project. As for the new chips, the company said it plans to install Azure Integrated HSMs in every new data center server starting next year. The timeline for the DPU roll-out, however, remains unclear at this stage.

Microsoft Ignite runs from November 19-22, 2024