NVIDIA H100 SXM in the flesh: 4nm Hopper GPU + 80GB HBM3 memory

NVIDIA's next-gen Hopper H100 SXM has been spotted in the flesh, and until now... we've only seen renders, so this is the first real pics of the H100 GPU.

2

The next-gen Hopper H100 SXM GPU was pictured in the flesh by Patrick Kennedy from ServeTheHome, with a device with a board model of "PG520". Inside, the NVIDIA H100 GPU is using TSMC's very latest CoWoS packaging technology, with a huge 814mm2 H100 GPU die with 6 memory modules around it: 80GB of ultra-fast HBM3 memory to be exact.

NVIDIA's new H100 GPU itself comes in two forms with two counts of CUDA cores: 16896 FP32 CUDA cores for the SXM variant and 14592 cores for the PCIe-based model. The model that we have here is the SXM variant, so we have the full 16896 FP32 CUDA cores and 80GB of HBM3 memory.

• Read more: You can buy NVIDIA's next-gen Hopper GPU + 80GB HBM2e for $36,550
• Read more: NVIDIA H100 GPU full details: TSMC N4, HBM3, PCIe 5.0, 700W TDP, more
• Read more: NVIDIA reveals next-gen Hopper GPU architecture, H100 GPU announced
• Read more: NVIDIA can sustain the world's internet traffic with 20 x H100 GPUs

2

We're looking at power consumption numbers of up to 700W, which is a big upgrade over the previous Ampere and Volta GPU architectures in data center GPU form: they use 400-450W, meaning H100 uses up to 250-300W more power.

• Read more: NVIDIA Hopper GPU is up to 40x faster with new DPX instructions
• Read more: NVIDIA is turning data centers into 'AI factories' with Hopper GPU
• Read more: NVIDIA Eos: the world's fastest AI supercomputer, 4608 x DGX H100 GPUs