Micron's new HBM3 Gen2 memory is the world's fastest with 1.2 TB/s of memory bandwidth

Micron has announced that it's currently sampling its new 24GB HBM3 Gen2 memory to its customers, with the new high-speed memory featuring 1.2 TB/s of memory bandwidth. Micron notes that HBM3 Gen2 delivers 2.5 times greater performance per watt than previous generations, making it ideal for AI data centers where performance, capacity, and efficiency are important.

2

The 8-high stack of 24GB of HBM3 Gen2 has increased pin speed to over 9.2Gb/s, a 50% improvement over existing HBM3 solutions. Over a 1024-bit bus, you've got 1.2 TB/s of bandwidth, which is a massive improvement over the current 820 GB/s found in existing HBM3.

With the current AI boom and race to see who can build the most supercomputers, this sort of thing will give Micron's flagship memory products an edge over the competition. With 24GB HBM3 Gen2 memory rolling out, Micron adds that 12-high stack 36GB capacity options will begin sampling in early 2024. And as part of its roadmap Micron lists 'HBMNext' as the next evolution beyond HBM3 Gen2, with 64GB capacity and over 2 TB/s of bandwidth arriving sometime around 2026.

"Micron's HBM3 Gen2 technology was developed with a focus on unleashing superior AI and high-performance computing solutions for our customers and the industry," said Praveen Vaidyanathan, vice president and general manager of Micron's Compute Products Group. "One important criterion for us has been the ease of integrating our HBM3 Gen2 product into our customers' platforms."

And with the following quote from NVIDIA, we assume that Team Green is one of the customers getting access to early HBM3 Gen2 samples.

"At the core of generative AI is accelerated computing, which benefits from HBM high bandwidth with energy efficiency," said Ian Buck, vice president of Hyperscale and HPC Computing at NVIDIA. "We have a long history of collaborating with Micron across a wide range of products and are eager to be working with them on HBM3 Gen2 to supercharge AI innovation."