Cerebras Systems has largest chip EVER with 1.2 TRILLION transistors

Cerebras Systems just unveiled the largest chip ever at Hot Chips 31, which is a A Symposium on High Performance Chips hosted yearly with the biggest companies in the tech space involved.

3

The new Cerebras Wafer Scale Engine chip from Cerebras has a huge 1.2 trillion (1,200,000,000,000) transistors compared to the largest GPU in NVIDIA's GV100 and its 21.1 billion (21,100,000,000). NVIDIA requires just 815 square millimeters of die space for its GV100 and 21.1 billion transistors, while Cerebras Systems requires a huge 46,225 square millimeters of silicon.

Cerebras' new chip is so big that in comparison photos, the company compares its new 1.2T transistor chip side-by-side with a freaking keyboard. Yes, this new chip is as big as a desktop keyboard.

3

Cerebras has to "break a lot of rules" to design its new Wafer Scale Engine chip, with the company building their own software tools to even begin desining the elaborate chip. Normally when you bake circuits into the wafer, there will be areas that are unusable. Cerebras couldn't have that, so this is where the rules being broken took place.

The company had to think ahead of this, building in redundant circuits that would route around the defects so that it could cram in 400,000 processor cores in a "miniature internet" that stays online when individual server systems go offline.

Cerebras has the exclusive rights to the IP that makes this process possible, even though TSMC is manufacturing the chip for them.

It will use up to 15kW of power which is crazy, but with 400K processor cores it makes sense -- while all those chips and the 1.2T transistors of tech cranking along will require a high-end, very complex watercooling system.

3

Cerebras Wafer Scale Engine was designed with deep learning workloads in mind, with Cerebras explaining its chip as a "wafer scale" implementation of a neural network. The new high-performance chip is already running customer workloads according to Cerebras, but no pricing has been unveiled. The company won't sell the chip on its own, but rather in a computer "appliance" it has designed.

This is a huge leap in computing power, and will see other companies scrambling to beat Cerebras in the coming years with its exciting new 1.2T chip and packaged system. It will also be interesting to see if any companies adopt this system, and where they'll be used.