OpenAI disbands team dedicated to mitigating existential risk of AI

The team at OpenAI that was dedicated to reducing the existential risk of the emerging technology has reportedly been disbanded, only days after two of the company's founders quit.

2

According to a new report from Wired, OpenAI's Superalignment team, which was tasked on identifying risks of advanced artificial intelligence-powered systems going rogue and endangering humanity, has been disbanded by the report, and absorbed into its numerous research and development ranks. The report states research into the risk of AI-powered systems will now be headed by OpenAI cofounder John Schulman.

Notably, Ilya Sutskever and Jan Leike, founders of OpenAI, announced only a few days ago they were departing the company. Both Sutskever and Leike were top researchers focussing on AI risk mitigation, and following his departure from the company, Leike took to X explaining he had been fighting with OpenAI leadership for quite some time about its core values but, unfortunately, reached his tipping point this week.

Furthermore, Leike said the Superaligment has been "sailing against the wind" and is struggling to get enough compute power to conduct critical research. In a nutshell, Leike said that OpenAI needs to be more focussed on security and safety of its AI models.

"Currently, we don't have a solution for steering or controlling a potentially superintelligent AI, and preventing it from going rogue. But humans won't be able to reliably supervise AI systems much smarter than us, and so our current alignment techniques will not scale to superintelligence. We need new scientific and technical breakthroughs," said the Superalignment team when it launched in July 2023