Microsoft unveils new Azure Cobalt CPU and Maia 100 GPU designed for AI and the Cloud

Microsoft formally announced its custom AI chips at the company's Ignite conference - the Microsoft Azure Cobalt 100 CPU built using Arm architecture and the Microsoft Azure Maia AI Accelerator, a GPU designed and optimized for generative AI and large language models. Both are set to arrive in 2024, and based on the current demand for NVIDIA hardware like the H100 GPUs, it could see Microsoft and its massive Azure cloud move away from having to buy pricey chips from other companies.

5

Microsoft's Maia 100 AI Accelerator, image credit: Microsoft.

The Azure Cobalt 100 CPU and Azure Maia 100 AI Accelerator have been developed in-house at Microsoft, alongside an overhaul of the entire Azure cloud server stack as the Age of AI begins. The Maia 100 AI Accelerator uses TSMCC's 5nm process with 105 billion transistors, which is higher than the NVIDIA Hopper H100 GPU's 80 billion. However, it is lower than the 153 billion found on AMD's MI300X AI GPU built on a similar 5nm process.

Of course, transistor counts and overall performance are two different things, but it's an impressive showing from Microsoft. And it's the very first liquid-cooled processor built by Microsoft.

"Microsoft is building the infrastructure to support AI innovation, and we are reimagining every aspect of our data centers to meet the needs of our customers," said Scott Guthrie, executive vice president of Microsoft's Cloud + AI Group. "At the scale we operate, it's important for us to optimize and integrate every layer of the infrastructure stack to maximize performance, diversify our supply chain, and give customers infrastructure choice."

5

A custom-built rack for the Maia 100 AI Accelerator, image credit: Microsoft.

The new Azure Cobalt 100 CPU and Maia 100 AI Accelerator chips are key to this initiative. These chips will be installed in Microsoft data centers to power the company's AI services like Copilot and OpenAI. OpenAI, one of the leaders in AI and large language models, provided feedback and aided Microsoft in developing the Maia AI Accelerator.

"Since first partnering with Microsoft, we've collaborated to co-design Azure's AI infrastructure at every layer for our models and unprecedented training needs," said Sam Altman, CEO of OpenAI. "We were excited when Microsoft first shared their designs for the Maia chip, and we've worked together to refine and test it with our models. Azure's end-to-end AI architecture, now optimized down to the silicon with Maia, paves the way for training more capable models and making those models cheaper for our customers."

5

Microsoft's Azure Cobalt 100 CPU, image credit: Microsoft.

Maia 100 is currently being tested on GPT 3.5 Turbo and is in the early phases of its deployment, so the company isn't sharing full specifications or benchmarks right now. Microsoft also notes that it will continue to partner, work with, and deploy hardware from companies like NVIDIA and AMD into AI systems.

The other chip, the Azure Cobalt 100 CPU, is built on Arm architecture with a 128-core design that features a 64-bit Armv9 instruction set - with Microsoft stating that it's able to deliver up to a 40% increase in performance compared to the current generation of Arm processors running the Azure cloud.