NVIDIA's full-spec Blackwell B200 AI GPU uses 1200W of power, up from 700W on Hopper H100

NVIDIA revealed its next-generation Blackwell B200 AI GPU at its recent GTC 2024 (GPU Technology Conference) event but left out some details that we're now discovering... like the new AI GPU consuming up to a whopping 1200W of power.

3

NVIDIA's new Blackwell B200 versus Hopper H100 AI GPUs (source: NVIDIA)

The new information on the Blackwell AI GPUs comes directly from NVIDIA SVP and GPU Architect, Jonah Albe, along with Ian Buck, the VP of Hyperscale and HPC at NVIDIA. Jonah pointed out that NVIDIA's new Blackwell GPU uses a completely different microarchitecture to Hopper, with Blackwell featuring 2nd Generation Transformer Engine Technology that adds both FP4 and FP6 compute formats, which along with new software optimizations that NVIDIA has made, unleashes Blackwell to be the fastest AI chip on the planet.

Blackwell has a 32% increase in FP64 compute performance with B200 versus Hopper H100, with Blackwell being an AI GPU first and foremost, FP64 compute performance isn't as important from an AI workload standpoint, where the lower you go, the faster the AI inferencing and training capabilities become.

• Read more: NVIDIA spent $10 billion on developing its next-generation Blackwell GPU
• Read more: NVIDIA CEO: our next-gen Blackwell GPU will cost $30,000 to $40,000 each
• Read more: NVIDIA GB200 Grace Blackwell Superchip: 864GB HBM3E, 16TB/sec bandwidth
• Read more: NVIDIA Blackwell AI GPU: multi-chip die, 208B transistors, 192GB HBM3E
• Read more: NVIDIA's new Blackwell-based DGX SuperPOD: ready for trillion-parameter scale for generative AI

NVIDIA moved to a chiplet (MCM or multi-chip module) design for Blackwell, improving performance through the use of chiplets versus improving yields on a monolithic GPU. Programmers could be worried of having to tweak their code for the chiplet -- or multi-chip -- design, but they won't need to worry. NVIDIA says that the GPUs don't know their separate, and act as one, driven by CUDA advancements that accelerate performance from the chiplet design.

Now, we might refer to NVIDIA's new Blackwell B200 as an "AI GPU," but CEO Jensen Huang doesn't. Jensen says that Blackwell is a platform that has a range of products based on GPUs. NVIDIA has three different Blackwell 'platforms', in Jensen's words, each with varying Blackwell GPU specs.

3

NVIDIA's new Blackwell B200 AI GPU (source: NVIDIA)

The full-spec Blackwell B200 features a maximum 1200W TDP, which is a 500W increase over the 700W that Hopper H100 consumes. NVIDIA's new GB200 Superchip will feature dual B200 AI GPUs and a Grace CPU that will consume 2700W in total. This is split into the 1200W x 2 from the B200 AI GPUs and 300W from the Grace CPU and I/O.

Under the full-spec B200 AI GPU, NVIDIA has B200 used inside of DGX and HGX platforms, which are optimized chips with 90% of the full-spec B200 variant, using around 1000W of power (200W less than full-spec B200). We don't know if NVIDIA is using a lower TDP on these B200 chips, or if they have cut-down specs versus the full-spec B200.

NVIDIA's last Blackwell GPU is the B100, which has a 700W TDP -- the same as Hopper H100 -- with 70% of the performance of the full-spec 1200W beast B200 AI GPU.