Cohere CEO says LLMs training each other is like a 'human centipede effect', Elon Musk agrees

Elon Musk's xAI startup plans huge breakthrough with Grok LLM models, Grok 3 training powered by 100,000 x NVIDIA H100 AI GPUs, Grok 2 coming in August.

2 minutes & 35 seconds read time

Elon Musk's AI startup is training its next-generation Grok 3 LLM model, which is being trained on a monster 100,000 x NVIDIA H100 AI GPUs.

In a new post on X, the SpaceX and Tesla founder talked about his xAI start-up, replying to @BasedBeffJezos on X who was replying to @BasedBeffJezos who posted a video from Aidan Gomez, the CEO and co-founder of Cohere, who said "models training on each other's data is like a human centipede effect".

This is where Musk replied, saying: "Sadly quite true. It takes a lot of work to purge LLMs from the internet training data. Grok 2, which comes out in August, will be a giant improvement in this regard". This is where @BasedBeffJezos said "Grok 2 in August will be Based AGI". Musk replied again, saying: "Grok 3 end of year after training on 100K H100s should be really something special".

You can see that video below:

Earlier this year, Tesla said it would be sending billions of dollars buying NVIDIA AI GPUs and AMD AI GPUs, so these numbers will radically change throughout the year as Tesla scoops up more AI silicon from NVIDIA. The recent $500 million investment into the Dojo Supercomputer is "only equivalent" to 10,000 x NVIDIA H100 AI GPUs, said Musk in January 2024, adding, "Tesla will spend more than that on NVIDIA hardware this year. The table stakes for being competitive in AI are at least several billion dollars per year at this point".

Buy at Amazon

NVIDIA H100 80 GB Graphic Card PCIe HBM2e Memory 350W (NVIDIA H100 80 GB)

TodayYesterday7 days ago30 days ago
Buy at Newegg
* Prices last scanned on 7/23/2024 at 4:39 am CDT - prices may not be accurate, click links above for the latest price. We may earn an affiliate commission.

Anthony joined the TweakTown team in 2010 and has since reviewed 100s of graphics cards. Anthony is a long time PC enthusiast with a passion of hate for games built around consoles. FPS gaming since the pre-Quake days, where you were insulted if you used a mouse to aim, he has been addicted to gaming and hardware ever since. Working in IT retail for 10 years gave him great experience with custom-built PCs. His addiction to GPU tech is unwavering and has recently taken a keen interest in artificial intelligence (AI) hardware.

Newsletter Subscription

Related Tags