Ex-Google chip designers launch MatX startup: will develop AI chips specifically for LLMs

A couple of ex-Google chip designers have left the US search giant, forming a new MatX startup to build AI processors specifically designed for LLMs (Large Language Models).

2

Mike Gunter and Reiner Pope used to work at Google, forming MatX, which has one objective: design next-generation silicon specifically for processing the data needed to fuel large language models (LLMs). LLMs are the basis in which the generative AI world sits on, with the likes of ChatGPT from OpenAI, Gemini from Google, and other LLM-powered generative AI platforms.

Gunter used to focus on designing hardware like chips to run AI software, while Pope wrote the AI software itself for Google. Google has been working hard at building its own in-house AI processors with Tensor Core Processors, first designed before LLMs became a thing and were too generic for the tasks at the time.

Pope said: "We were trying to make large language models run faster at Google and made some progress, but it was kind of hard. Inside of Google, there were lots of people who wanted changes to the chips for all sorts of things, and it was difficult to focus just on LLMs. We chose to leave for that reason".

• Read more: Microsoft, OpenAI team up for $100B AI supercomputer codenamed Stargate

NVIDIA has been absolutely dominating the AI silicon market for a while now, with its current-gen Hopper H100 and upcoming H200 AI GPUs doing extremely well, while the next-generation Blackwell B200 AI GPU has been announced and will be unleashed later this year.

NVIDIA's fleet of AI GPUs are fantastic at handling oodles and oodles of small tasks, and the company bet big on the future of CUDA and AI software over 10 years ago. The company breaks up the real estate on its GPUs to handle various computing jobs, including vast amounts of data around the chip, including HBM memory. Some of NVIDIA's design choices can also be stuck with the "past eras" of computing rather than the AI boom, and they have performance tradeoffs.

The new MatX founders think that extra real estate on the GPU adds unneeded costs and massive complexity in the new era of AI. MatX is doing things differently, where they'll be designing silicon with one single large processing core aimed at multiplying numbers as quickly as possible -- which is what LLMs require -- and the company is betting big on this future.

Pope said: "NVIDIA is a really strong product and clearly the right product for most companies, but we think we can do a lot better".