Cohere CEO says LLMs training each other is like a 'human centipede effect', Elon Musk agrees

Elon Musk's AI startup is training its next-generation Grok 3 LLM model, which is being trained on a monster 100,000 x NVIDIA H100 AI GPUs.

In a new post on X, the SpaceX and Tesla founder talked about his xAI start-up, replying to @BasedBeffJezos on X who was replying to @BasedBeffJezos who posted a video from Aidan Gomez, the CEO and co-founder of Cohere, who said "models training on each other's data is like a human centipede effect".

This is where Musk replied, saying: "Sadly quite true. It takes a lot of work to purge LLMs from the internet training data. Grok 2, which comes out in August, will be a giant improvement in this regard". This is where @BasedBeffJezos said "Grok 2 in August will be Based AGI". Musk replied again, saying: "Grok 3 end of year after training on 100K H100s should be really something special".

You can see that video below:

• Read more: Dell is building an AI factory with NVIDIA AI GPUs to train Grok for xAI and Elon Musk
• Read more: Elon Musk says training next-gen Grok 3 will require 100,000 NVIDIA H100 AI GPUs

Earlier this year, Tesla said it would be sending billions of dollars buying NVIDIA AI GPUs and AMD AI GPUs, so these numbers will radically change throughout the year as Tesla scoops up more AI silicon from NVIDIA. The recent $500 million investment into the Dojo Supercomputer is "only equivalent" to 10,000 x NVIDIA H100 AI GPUs, said Musk in January 2024, adding, "Tesla will spend more than that on NVIDIA hardware this year. The table stakes for being competitive in AI are at least several billion dollars per year at this point".

• Read more: Tesla will spend billions of dollars on NVIDIA AI GPUs this year
• Read more: Tesla's Dojo AI supercomputer boss leaves, former Apple exec now leads Dojo
• Read more: Tesla's insane new Dojo D1 AI chip, a full transcript of its unveiling
• Read more: NVIDIA commands 90% of AI GPU market, competitors 'years from catching up'
• Read more: NVIDIA AI GPU shipments expected to surge 150% year-over-year in 2024