Meta using over 100,000 NVIDIA H100 AI GPUs for Llama 4, Zuck: 'bigger than anything I've seen'

Mark Zuckerberg has provided a small update on Meta's work on its new Llama 4 model, which is being trained on a cluster of AI GPUs "bigger than anything" Zuck has seen.

2

Meta is cooking its new Llama 4 right now, with Zuckerberg telling investors and analysts on an earnings call this week that the initial launch of Llama 4 is expected later this year. Zuck said: "We're training the Llama 4 models on a cluster that is bigger than 100,000 H100s, or bigger than anything that I've seen reported for what others are doing. I expect that the smaller Llama 4 models will be ready first".

Meta's new AI supercomputer with its 100,000+ NVIDIA H100 AI GPUs reportedly cost over $2 billion for the H100 AI GPU chips alone, which means Mark Zuckerberg is signing some fat cheques to NVIDIA. Speaking of which, NVIDIA CEO Jensen Huang recently spoke with Meta CEO Mark Zuckerberg, where Jensen said Meta now has 600,000+ NVIDIA H100 AI GPUs to which Zuck replied saying that Meta were "good customers for NVIDIA".

• Meta nearly finished with 100,000+ NVIDIA H100 AI GPU cluster: online in October or November

Meta is in a race with xAI and Elon Musk, with Musk's xAI to double the size of its Colossus AI supercomputer cluster, which has 100,000 NVIDIA Hopper AI GPUs and is getting upgraded to 200,000 NVIDIA Hopper AI GPUs. But the fight of the biggest AI supercluster is damn interesting to watch, just don't show me the power bills.

• Read more: NVIDIA CEO: Meta has 600K H100 AI GPUs, Meta 'good customers for NVIDIA'
• Read more: Meta's long-term vision for AGI: 600,000 x NVIDIA H100-equivalent AI GPUs
• Read more: Elon Musk's new Memphis Supercluster uses portable power generators
• Read more: Elon turns on xAI's new AI supercomputer: 100K liquid-cooled H100 AI GPUs
• Read more: Meta's huge 16,384 NVIDIA H100 cluster: HBM3 crashed half of Llama 3 training
• Read more: NVIDIA to make $210B revenue from selling its Blackwell GB200 AI servers in 2025 alone