NVIDIA officially teases next-gen B100 Blackwell GPU: over 4x as fast as H100 AI GPU

NVIDIA's next-generation B100 Blackwell GPU has been teased, and this time it's not coming from a leaker... but from NVIDIA themselves, teasing 4x the AI inference performance over its already industry-leading H100 Hopper AI GPU.

NVIDIA B100 AI GPU against A100, H100, H200 in AI inferencing (source: NVIDIA)

At the recent SuperComputing 2023 Special Address, NVIDIA provided some slides that compare its A100 Ampere AI GPU against their H100 and H200 Hopper AI GPUs, against the next-gen B100 Blackwell AI GPU with some mind-blowing results. NVIDIA showed GPT-3 175B inference performance where 1x is the baseline with A100, leading an 11x increase with the H100, a larger 18x increase with the H200, which will be released in 2024, and then it looks like a gigantic leap with B100 in 2024.

I've edited the photo and used the 11x performance with H100, showing that NVIDIA is teasing at least 3x the performance in GPT-3 175B inference performance with its next-gen B100 Blackwell AI GPU.

As for what we can expect inside of the next-gen B100 Blackwell GPU is a mind-blowing 178 billion transistors... over double the transistors, as well as packing the latest HBM3e memory from Micron, just like its just-announced H200 Hopper AI GPU that will arrive in 2024. The new H200 AI GPU supports up to 141GB of HBM3e memory with up to 4.8TB/sec of memory bandwidth, so we should expect similar memory configurations with B100 AI GPUs.

• Read more: NVIDIA's next-gen B100 'Blackwell' AI GPUs hit supply chain certification stage
• Read more: NVIDIA flexes H100 AI GPU muscle, setting new records in AI benchmarks
• Read more: NVIDIA's next-gen GB200 'Blackwell' GPU listed on its 2024 data center roadmap

Blackwell = chiplet GPU teased: we are hearing about NVIDIA utilizing a chiplet design with its next-gen B100 Blackwell GPU, a first for NVIDIA, competing against AMD's upcoming Instinct MI300 accelerator. With a rumored 178 billion transistors, NVIDIA will push silicon to its limits, and I can't wait.

NVIDIA uses the Blackwell GPU architecture name because of the American statistician and mathematician, recognized for his contributions to game theory, probability, and information theories, David Harold Blackwell. After Blackwell, we have a successor to Blackwell in 2025/2026, so we don't have too much longer to wait for some next-next-gen GPUs in both the AI and gaming GPU worlds.

5

NVIDIA's new H200 Hopper AI GPU (source: NVIDIA)

NVIDIA's new H200 Hopper GPU was also announced at the same time as teasing its next-gen B100 Blackwell GPU, with the new H200 coming in 2024 with the world's fastest HBM3e memory from Micron. NVIDIA also announced that its Grace Hopper Superchips would power the new Exaflop Jupiter supercomputer.

The new NVIDIA H200 GPUs feature Micron's latest HBM3e memory, with capacities of up to 141GB per GPU with up to 4.8TB/sec of memory bandwidth. This is 1.8x more memory capacity than the HBM3 memory on H100, and up to 1.4x more HBM memory bandwidth over H100. NVIDIA uses either 4x or 8 x H200 GPUs for its new HGX H200 servers, so you're looking at a huge 1.1TB of HBM3e memory with up to 4.8TB/sec of memory bandwidth. Impressive numbers, NVIDIA.

NVIDIA also promises that its new H200 AI GPUs will be compatible with its existing HGX H100 systems, which makes it easy for customers to upgrade their systems. NVIDIA has partners like ASUS, ASRock Rack, Dell, Eviden, GIGABYTE, Hewlett Packard Enterprise, Ingrasys, Lenovo, QCT, Wiwynn, Supermicro, and Wistron. NVIDIA will have updated solutions when the H200 GPUs are made available in Q2 2024.