NVIDIA's new Grace CPU Superchip: 144-core CPU, 600GB of GPU memory

NVIDIA unleashed its next-gen Hopper GPU architecture and new flagship NVIDIA H100 GPU, but also its next-gen Grace CPU design that has been unveiled alongside two new Superchip platforms.

3

The new NVIDIA Grace Hopper Superchip has been designed for giant-scale AI and HPC (high-performance computing) use, we have NVIDIA's in-house Grace CPU + Hopper GPU combo, with an insane 600GB of GPU memory, 900GB/sec of coherent NVLink interface, and it runs the NVIDIA Computing Stack software suite.

There's a huge 30x system memory bandwidth to the GPU improvement over the DGX A100 here, with a CPU and GPU that were designed for giant-scale AI and HPC. The 900GB/sec of coherent interface is 7x faster than PCIe 5.0 which is only just dropping into gamers' desktops now, with PCIe 5.0-enabled GPUs launching this year.

• CPU+GPU designed for giant-scale AI and HPC
• New 900 gigabytes per second (GB/s) coherent interface, 7X faster than PCIe Gen 5
• 30X higher aggregate system memory bandwidth to GPU compared to DGX A100
• Runs all NVIDIA software stacks and platforms, including NVIDIA HPC, NVIDIA AI, and NVIDIA Omniverse

3

• High-performance CPU for HPC and cloud computing
• Super chip design with up to 144 Arm v9 CPU cores
• World's first LPDDR5x with ECC Memory, 1TB/s total bandwidth
• SPECrate2017_int_base over 740 (estimated)
• 900 GB/s coherent interface, 7X faster than PCIe Gen 5
• 2X the packaging density of DIMM-based solutions
• 2X the performance per watt of today's leading CPU
• Runs all NVIDIA software stacks and platforms, including RTX, HPC, AI, and Omniverse

NVIDIA also unveiled the second Superchip platform which has the Grace CPU, packing high-performance Grace ARM CPU cores that are also built for next-gen HPC and AI systems. There's 144 cores based on the ARM Neoverse architecture, pushing 740 SPECrate@2017_int_base performance.

That might not sound like much, but based on previous estimates on performance these numbers were at the 300-ish level, and now we're pushing 740. That's a 2.5x increase since NVIDIA first unveiled its new Grace CPU architecture.

• Read more: NVIDIA reveals next-gen Hopper GPU architecture, H100 GPU announced
• Read more: NVIDIA can sustain the world's internet traffic with 20 x H100 GPUs
• Read more: NVIDIA Hopper GPU is up to 40x faster with new DPX instructions
• Read more: NVIDIA is turning data centers into 'AI factories' with Hopper GPU
• Read more: NVIDIA Eos: the world's fastest AI supercomputer, 4608 x DGX H100 GPUs

NVIDIA's new Grace CPU not only packs 144 ARM v9 CPU cores, but it's the world's first with LPDDR5X memory, with 1TB of memory bandwidth. The new Grace CPUs will chew through some power, with 500W of power expected to be required.