Google finds hackers using AI to discover and develop a zero-day exploit for the first time for mass attacks

With AI models achieving new milestones, it was inevitable that security experts' fears would come true. Google's Threat Intelligence Group has, for the first time, discovered a threat actor using a zero-day exploit likely developed with AI. A zero-day vulnerability is a software or hardware flaw that is unknown to developers, leaving them with zero days to patch it before attackers can exploit it.

The exploit targeted a popular open-source web-based system administration tool. Google called the threat actor a prominent cybercrime group that allegedly planned to use the flaw in a mass exploitation campaign. Had it gone undetected, the flaw would have allowed hackers to bypass two-factor authentication and access victim accounts with just a password.

Google investigators suspect, with "high confidence," that the exploit was developed with the help of an unidentified AI program, based on the Python code's structure and content. The report says the script has educational docstrings, a fake CVSS score, and a Pythonic format typical of LLM training data.

"For example, the script contains an abundance of educational docstrings, including a hallucinated CVSS score, and uses a structured, textbook Pythonic format highly characteristic of LLMs training data (e.g., detailed help menus and the clean _C ANSI color class)," the report reads.

2

The company didn't go into details about how the attack was uncovered, to avoid helping hackers, but ruled out the possibility that Gemini was involved in the process. Google also did not name the bad actors, but noted groups tied to China and North Korea have shown a strong interest in using AI to exploit vulnerabilities.

In an interview with The New York Times, GTIG chief analyst John Hultquist called the exploit "a taste of what's to come." "We believe this is the tip of the iceberg. Other AI-developed zero-days are probably out there," he added.

The Google attack is alarming, but defenders aren't sitting idle either. AI companies have released cutting-edge models, such as Anthropic's Claude Mythos, which helps businesses discover and patch high-severity vulnerabilities before they are exploited. Google also said that 'AI can also be a powerful tool for defenders.' Still, there is no denying that it is only a matter of time before AI becomes just as powerful a weapon for cybercriminals as it is for those defending against them.