In an unsettling sign of things to come, Google has uncovered evidence that hackers used an AI program to discover a previously unknown software vulnerability that could have been exploited at mass scale, marking what the company believes is the first confirmed case of an AI-developed zero-day exploit.
What Was Found
According to a recent Google report examining AI-related cyber threats, the zero-day vulnerability targeted a widely used open-source, web-based system administration tool. Specifically, the flaw would have allowed attackers to bypass two-factor authentication, enabling account access with only a password and thus eliminating the secondary verification code typically sent to a user’s phone or email.
Google’s investigators concluded the exploit was likely AI-assisted based on distinctive characteristics of the underlying code. As the report notes, the script contained an abundance of educational docstrings, including a hallucinated CVSS score, and followed a structured, textbook Pythonic format highly characteristic of large language model training data, such as detailed help menus and clean ANSI color class implementations.
The Threat Actor’s Intent
The criminal actor behind the exploit had planned to deploy it in a mass exploitation event, and Google’s proactive discovery of the vulnerability may have prevented that from occurring. The company also observed cybercriminal groups attempting to collaborate on exploiting the flaw, though it has declined to disclose specifics of how the investigation unfolded, a deliberate choice to avoid revealing details of its own detection methods.
Google’s chief threat intelligence analyst, John Hultquist, acknowledged that the public would naturally want more granular details, but affirmed that the company has sound reasons for withholding certain data. He also noted that the disruption may have been aided by other, undisclosed challenges the threat actor was already facing.
How Google Responded
Fortunately, Google detected the vulnerability before it could be weaponized and coordinated with the affected vendor to responsibly disclose and patch it. Notably, Google’s own Gemini chatbot played no role in discovering the flaw.
The Bigger Picture
Despite the successful intervention, Google is clear-eyed about the broader implications. Hultquist warned that this incident is likely only the beginning, as other AI-developed zero-days are probably already in circulation. He further cautioned that if criminal actors are leveraging AI in this way, well-resourced state-sponsored actors almost certainly are too.
This concern is not hypothetical. Last November, Anthropic reported that suspected Chinese state-sponsored hackers had attempted to use its AI coding tool to compromise approximately 30 targets globally.
On the defensive side, AI developers are racing to stay ahead. Anthropic, for instance, has been developing advanced models such as Mythos specifically to help vendors identify and patch vulnerabilities before they can be exploited. Even so, the trajectory suggests an accelerating arms race between AI-powered attackers and defenders, one in which the balance of advantage remains deeply uncertain.