Meta has two new AI data centers equipped with over 24,000 NVIDIA H100 GPUs

We know that AI is big business, and that is why companies like Microsoft, Meta, Google, and Amazon are investing mind-boggling amounts of money in creating new infrastructure and AI-focused data centers. As per Meta's latest post regarding its "GenAI Infrastructure," the company has announced two "24,576 GPU data center scale clusters" to support current and next-gen AI models, research, and development.

2

Meta's AI infrastructure, image credit: Meta.

That's over 24,000 NVIDIA Tensor Core H100 GPUs, with Meta adding that its AI infrastructure and data centers will house 350,000 NVIDIA H100 GPUs by the end of 2024. There's only one response to seeing that many GPUs: a comically long and cartoonish whistle or a Neo-style "Woah." Meta is going all in on AI, a market in which it wants to be the leader.

"To lead in developing AI means leading investments in hardware infrastructure," the pot writes. "Meta's long-term vision is to build artificial general intelligence (AGI) that is open and built responsibly so that it can be widely available for everyone to benefit from."

The two GPU-heavy clusters feature a slightly different network setup; however, both have 400Gbps interconnect capability. One features Meta's network fabric solution based on the Arista 7800 (with Wedge400 and Minipack2 OCP rack switches), while the other cluster features NVIDIA Quantum2 fabric. It's a more cutting-edge solution than Meta's AI Research SuperCluster (RSC), featuring 16,000 NVIDIA A100 GPUs from 2022.

"The efficiency of the high-performance network fabrics within these clusters, some of the key storage decisions, combined with the 24,576 NVIDIA Tensor Core H100 GPUs in each, allow both cluster versions to support models larger and more complex than that could be supported in the RSC and pave the way for advancements in GenAI product development and AI research," Meta explains.

Meta plans to continue rapidly growing and evolving its AI infrastructure, "We are constantly evaluating and improving every aspect of our infrastructure, from the physical and virtual layers to the software layer and beyond. Our goal is to create systems that are flexible and reliable to support the fast-evolving new models and research."