Researchers create never-before-seen cyberattack using generative AI

Researchers are warning that it's only a matter of time before AI-powered malware, such as the "worm" they created, is discovered in the wild.

2

The team behind the new malware published a paper that is yet-to-be-peer-reviewed that details the creation of a new type of malware that targets AI-powered email assistants. The researchers conducted their experiment in a closed-circuit environment and found that their malware, or "worm," was capable of targeting email assistants powered by popular language models such as OpenAI's GPT-4, Google's Gemini Pro, and LLaVA.

The result was the worm infecting these email assistants, obtaining sensitive user information, and then sending out spam emails that can infect other PCs, replicating the same process. So, how does it work? The researchers used an "adversarial self-replicating prompt" that forces the targeted AI model to create another prompt within its response. Essentially, the target AI assistant receives a prompt, generates a response, and within that response is another prompt.

This method creates a domino effect, and since these AI assistants contain large amounts of sensitive information, the researchers designed it to retrieve any information that could be considered confidential. Examples of confidential information that the worm successfully grabbed were names, telephone numbers, credit card numbers, social security numbers, and more.

"The generated response containing the sensitive user data later infects new hosts when it is used to reply to an email sent to a new client and then stored in the database of the new client," Nassi told Wired

The worries don't stop there as the now-infected database is then told by the malware to send confidential information to a new client, which secretly contains a new worm to infect that database. This infected message can even be in the form of an image, which the team successfully performed.

"By encoding the self-replicating prompt into the image, any kind of image containing spam, abuse material, or even propaganda can be forwarded further to new clients after the initial email has been sent," Nassi added