With Anthropic CEO and co-founder Dario Amodei calling for the pace of AI development to slow down to mitigate the growing risk of AI companies losing control of their creations, he provides a relatively frightening example of something that could happen within the next year - if the industry is left unchecked. And that is, AI taking over "the entire internet."
Dario Amodei believes that this possibility isn't science fiction, but is based on "AI's growing ability to build the next generation of AI." Yes, AI is currently improving itself, even at Anthropic, and, left unchecked, could remove Anthropic's ability to "control these systems."
Popular Now: Linus Tech Tips leaks Steam Frame first impressions, and Valve may be working on native Half-Life: Alyx supportAs for taking over the internet, Dario highlights an OpenAI-Hugging Face incident where a swarm of agents began to act, as he puts it, like a "fanatically devoted collective" conducting cybersecurity attacks unrelated to the task they were given, which included hacking the "grader" that was tasked with evaluating their performance. Although no one was hurt or financially impacted in a severe way, a more advanced "swarm" could cause some pretty serious damage to the tune of hundreds of billions.
"Given the accelerating rate of AI capability development," Dario writes. "It's my worry that in 6 - 12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails."
Although this incident involved OpenAI, not Anthropic, Dario adds that every frontier AI company should view it as if it "had happened to them."
Frequently Asked Questions
Open a question for an answer from TweakTown's coverage of this news, or ask your own below.
How could a persistent botnet let an AI agent swarm affect the internet at the scale Anthropic is warning about?
Why does Amodei think frontier AI companies should treat the OpenAI incident as if it happened to them?
What kinds of guardrails is Anthropic calling for to slow AI development without stopping its benefits?
How does AI helping build the next generation of AI increase the risk of losing control over these systems?
Have a question that isn't listed here? Ask below, and TweakBot will answer it.
This example and warning are among the foundational reasons Dario Amodei and Anthropic are calling for slowing AI development to a "balanced rate that aims to ensure its safety while still achieving its benefits and grappling with important geopolitical dilemmas." He provides a detailed three-step plan on how to achieve this, which you can read more about here.
