NVIDIA can sustain the world's internet traffic with 20 x H100 GPUs

NVIDIA has just announced its next-gen Hopper GPU architecture, with the new flagship NVIDIA H100 GPU rocking 80 billion transistors, next-gen HBM3 memory technology, PCIe 5.0 connectivity, and enough power that if you use 20 of NVIDIA H100 GPUs you'd be able to sustain the entire world's internet traffic.

3

Yeah, you read that right: 20 x NVIDIA H100 GPUs have almost 5TB/sec of external connectivity, as well as 3TB/sec of memory bandwidth thanks to its HBM3 memory, which is enough that in the words of NVIDIA that 20 x NVIDIA H100 GPUs "can sustain the equivalent of the entire world's internet traffic".

In the middle of this is the new Transformer Engine -- which, when compared to the previous-gen A100 -- the new NVIDIA H100 speeds up natural language processing (Transformer being one of the most important deep learning models ever invented) by up to a gigantic 6x performance without losing accuracy. Beautiful work, NVIDIA... as always, hopping over the competition (which is yourself at this level).

• Read more: NVIDIA reveals next-gen Hopper GPU architecture, H100 GPU announced
• Read more: NVIDIA Hopper GPU is up to 40x faster with new DPX instructions
• Read more: NVIDIA announces new DGX H100 system: 8 x Hopper-based H100 GPUs
• Read more: NVIDIA is turning data centers into 'AI factories' with Hopper GPU
• Read more: NVIDIA Eos: the world's fastest AI supercomputer, 4608 x DGX H100 GPUs

The new Hopper GPU architecture from NVIDIA was named after pioneering US computer scientist, Grace Hopper, succeeding the Ampere GPU architecture. We have 80 billion transistors on the NVIDIA H100, which now becomes the world's largest and most powerful accelerator.

3

3

NVIDIA and TSMC (Taiwan Semiconductor Manufacturing Company) worked on the TSMC 4N process node, tweaking it for the Hopper-based H100 GPU. There's PCIe 5.0 support, next-gen ultra-fast HBM3 memory technology with an absolute blistering 3TB/sec (3000MB/sec). Insanity.

But there's also 4th-Generation NVIDIA NVLink technology here too, where you can use a new external NVLink Switch to extend NVLink as a scale-up networking beyond the server. You can connect a bonkers 256 x NVIDIA H100 GPUs at 9x higher bandwidth versus previous-gen GPUs using NVIDIA HDR Quantum InfiniBand.