AMD, Arm, Intel, Meta, Microsoft, NVIDIA, and Qualcomm are teaming up for next-gen AI standards

You'll not often see a headline like this, where all of the big tech companies are coming together for something, so be sure to let it sit for a minute. Imagine it's a DC or Marvel situation, and the Justice League or the Avengers are coming together to save the world. Of course, this is nothing like that, but (deep breath) AMD, Arm, Intel, Meta, Microsoft, NVIDIA, and Qualcomm are coming together to standardize next-generation narrow precision data formats for AI.

2

AMD, Arm, Intel, Meta, Microsoft, NVIDIA, and Qualcomm are teaming up.

This sounds a little boring compared to what a team-up like this could mean, but it is important for the future of AI training and inference. Narrow-precision AI data formats have played a major role in optimizing LLMs and AI, where the move from 32-bit floating point precision to the current 8-bits of precision played a big role in the evolution of AI tools and hardware.

Accelerating training and inference times, AI models can then take up less space and use less data and memory for improved performance and efficiency. The good news is AMD, Arm, Intel, Meta, Microsoft, NVIDIA, and Qualcomm have formed the Microscaling Formats (MX) Alliance to standardize the next-generation of 6 and 4-bit data types for AI.

With a name like the Microscaling Formats (MX) Alliance, it may be a bit like the Avengers. According to the announcement, the initial MX specs are already introducing "four concrete floating point and integer-based data formats (MXFP8, MXFP6, MXFP4, and MXINT8) compatible with current AI stacks." The Microscaling Formats (MX) Specification v1.0 is an open, license-free spec available through the Open Compute Project Foundation (OCP) "to enable and encourage broad industry adoption."

And with that, spokespeople from AMD, Arm, Intel, Meta, Microsoft, NVIDIA, and Qualcomm have taken the time to celebrate this democratization of AI. Here are a few of the quotes.

"AMD is pleased to be a founding member of the MX Alliance and has been a key contributor to the OCP MX Specification v1.0. This industry collaboration to standardize MX data formats provides an open and sustainable approach to continued AI innovations while providing the AI ecosystem time to prepare for the use of MX data formats in future hardware and software. AMD is committed to driving forward an open AI ecosystem and is happy to contribute our research results on MX data formats to the broader AI community."

Michael Schulte, Sr. Fellow, AMD

"The OCP MX spec is the result of a fairly broad cross-industry collaboration and represents an important step forward in unifying and standardizing emerging sub-8bit data formats for AI applications. Portability and interoperability of AI models enabled by this should make AI developers very happy. Benefiting AI applications should see higher levels of performance and energy efficiency, with reduced memory needs."

Pradeep Dubey, Senior Fellow and Director of the Parallel Computing Lab, Intel

"To keep pace with the accelerating demands of AI, innovation must happen across every layer of the stack. The OCP MX effort is a significant leap forward in enabling more scalability and efficiency for the most advanced training and inferencing workloads. MX builds upon years of internal work, and now working together with our valued partners, has evolved into an open standard that will benefit the entire AI ecosystem and industry."

Brian Harry, Technical Fellow, Microsoft

"The OCP MX specification is a significant step towards accelerating AI training and inference workloads with sub-8-bit data formats. These formats accelerate applications by reducing memory footprint and bandwidth pressure, also allowing for innovation in math operation implementation. The open format specification enables platform interoperability, benefiting the entire industry."

Paulius Micikevicius, Senior Distinguished Engineer, NVIDIA