Cerebras WSE-3 wafer-scale AI chip: 57x bigger than largest GPU with 4 trillion transistors

Cerebras Systems unveils its new WSE-3 AI chip that's 57x larger than NVIDIA H100 AI GPU, with 4 trillion transistors, and 900,000 AI-optimized cores.

2 minutes & 39 seconds read time

Cerebras Systems has just revealed its third-generation wafer-scale engine (WSE) chip, WSE-3, which packs 4 trillion transistors and 900,000 AI-optimized cores.

Cerebras WSE-3 wafer-scale AI chip: 57x bigger than largest GPU with 4 trillion transistors 1402

The company hasn't stopped on its journey of AI processor releases, with some truly crazy specifications for Cerebras' new WSE-3 chip. We have 4 trillion transistors, 900,000 AI-optimized cores, 125 petaflops of peak AI performance, and 44GB of on-chip SRAM made on the 5nm process node at TSMC.

WSE-3 also features either 1.5TB, 12TB, or 1.2PB of external memory -- yeah, 1.2 petabytes of memory -- capable of training AI models with up to 24 trillion parameters. Cerebras says its new WSE-3 has a die size of 46,225mm2, which is an insane 57x larger than NVIDIA's current H100 AI GPU, which measures 826mm2.

NVIDIA's current H100 AI GPU features 16,896 cores and 528 Tensor Cores, but the new Cerebras WSE-3 features an insane 900,000 AI-optimized cores (per chip), which is a massive 52x increase over the H100. Incredible stuff, truly incredible stuff.

Cerebras WSE-3 wafer-scale AI chip: 57x bigger than largest GPU with 4 trillion transistors 1403

Now, let's jump into the 21 petabytes per second of memory bandwidth... which is a mind-boggling 7000x more than the H100, with 214 petabits per second of fabric bandwidth (3715x faster than H100). Cerebras slaps 44GB of on-chip memory, which is 880x more than NVIDIA's current H100 AI GPU. Every number Cerebras has, puts its competitors to shame.

How much is the new WSE-3 AI chip? That's something we don't know, but don't expect them to be cheap.

Andrew Feldman, CEO and co-founder of Cerebras explained: "When we started on this journey eight years ago, everyone said wafer-scale processors were a pipe dream. We could not be more proud to be introducing the third-generation of our groundbreaking water scale AI chip. WSE-3 is the fastest AI chip in the world, purpose-built for the latest cutting-edge AI work, from mixture of experts to 24 trillion parameter models. We are thrilled for bring WSE-3 and CS-3 to market to help solve today's biggest AI challenges".

Cerebras WSE-3 specification highlights:

  • 4 trillion transistors
  • 900,000 AI cores
  • 125 petaflops of peak AI performance
  • 44GB on-chip SRAM
  • 5nm TSMC process
  • External memory: 1.5TB, 12TB, or 1.2PB
  • Trains AI models up to 24 trillion parameters
  • Cluster size of up to 2048 CS-3 systems

Rick Stevens, Argonne National Laboratory Associate Laboratory Director for Computing, Environment and Life Sciences said: "As a long-time partner of Cerebras, we are interested to see what's possible with the evolution of wafer-scale engineering. CS-3 and the supercomputers based on this architecture are powering novel scale systems that allow us to explore the limits of frontier AI and science. Cerebras' audacity continues to pave the way for the future of AI".

Buy at Amazon

NVIDIA H100 80 GB Graphic Card PCIe HBM2e Memory 350W

TodayYesterday7 days ago30 days ago
Buy at Newegg
* Prices last scanned on 4/12/2024 at 4:44 am CDT - prices may not be accurate, click links above for the latest price. We may earn an affiliate commission.
NEWS SOURCES:wccftech.com, cerebras.net

Anthony joined the TweakTown team in 2010 and has since reviewed 100s of graphics cards. Anthony is a long time PC enthusiast with a passion of hate for games built around consoles. FPS gaming since the pre-Quake days, where you were insulted if you used a mouse to aim, he has been addicted to gaming and hardware ever since. Working in IT retail for 10 years gave him great experience with custom-built PCs. His addiction to GPU tech is unwavering and has recently taken a keen interest in artificial intelligence (AI) hardware.

Newsletter Subscription

Related Tags