NVIDIA H100 AI GPUs inside of data centers use as much electricity as Guatemala and Lithuania

NVIDIA's current-gen H100 AI GPU is deployed across the world in countless data centers, supercomputers, and AI platforms... with over 3.5 million units expected to ship in 2024.

2

NVIDIA powers the Jupiter supercomputer with its new H200 AI GPU (source: NVIDIA)

Market analysis source, Stock Talk, has published an estimated figure on the power consumption of all of these NVIDIA H100 AI GPUs in the wild. In the tweet, it said: "By next year, NVIDIA's ~3.5 million units of deployed H100 GPUs will consume a whopping 13,000 GWh of electricity annually, which is greater than the power consumption of some entire countries like Guatamela and Lithuania".

• Read more: NVIDIA H200 AI GPU: up to 141GB of HBM3e memory with 4.8TB/sec bandwidth

This is something we have heard before, where crypto mining installations were using as much power as countries, where I reported back in 2020 that one particular Bitcoin mining facility in Kazakhstan was using enough power for 180,000 homes. Another story in 2021 pointed to the fact that Bitcoin mining was using more electricity than countries like the Netherlands and Argentina.

NVIDIA's extensive market share (90%+) of the AI GPU business has seen its AI GPUs used across the planet, and that's just for the A100 and H100 AI GPUs. NVIDIA has its upcoming refreshed H200 AI GPU, and then in 2024 and beyond, we'll have the new Blackwell GPU architecture and B100 AI GPU to look forward to. B100 will have improved power consumption, and magnitudes more AI power up for grabs.

If the power numbers are bad right now, NVIDIA is expected to ship somewhere between 1.5 and 2 million new AI GPUs in 2024, so we should expect those power numbers and panic to skyrocket by 3x the power numbers in 2024 with all those new AI GPUs being turned on.

• Read more: NVIDIA teases next-gen B100 Blackwell GPU: over 4x as fast as H100 AI GPU
• Read more: NVIDIA flexes H100 AI GPU muscle, setting new records in AI benchmarks
• Read more: AI platform floats in the ocean with 10,000 x NVIDIA H100 AI GPUs

NVIDIA's Grace Hopper Superchips (GH200) will be powering the Jupiter supercomputer, which is located at the Forschungszentrum Jülich facility in Gemany, and acts as part of the EuroHPC Joint Undertaking and contracted to Eviden and ParTec. The new Jupiter supercomputer will be used for Material Science, Climate Research, Drug Discovery, and more

Inside the new Jupiter supercomputer, it defines a new class of supercomputers that are designed to propel AI for scientific discovery, with a huge 93 ExaFLOPS of AI performance thanks to its huge 24,000 x GH200 Grace Hopper Superchips, 1.0 EF delivered HPC performance, Quantum-2 InfiniBand, 1.2PB/sec of aggregate bandwidth, with 18.2mW of power consumption