Browsers are Wide Open to LLM Prompt Injection Attacks

With the installation of just one malicious browser extension, or even the hijacking of one legitimate extension that’s already present, your systems could be wide open to the use of LLMs to stealthily steal data. A new class of prompt injection attacks that stretches across all browser types and generative AI (GenAI models) makes it trivial for attackers, abusing an existing browser permission that these models already have.

The prompt injection attacks have been articulated by LayerX, and the researchers demonstrate two of these “Man In The Prompt” proof-of-concept hacks that compromise ChatGPT and Google Gemini. Nearly all of the major LLM models are similarly vulnerable, however, and may be plugged into a broad assortment of data that attackers could trivially steal without being detected.

Vulnerability can exploit both internal and external LLMs

Potentially any type of browser extension can be created (or altered) to facilitate the prompt injection attacks, and the attackers can either plant command-and-control locally or use something hosted remotely. The most basic version of the attack is very simple: the extension will open background browser tabs through which it issues commands to the LLM, exfiltrate the contents of the chat to a log, then delete the chat before the user can check the LLM’s history and become aware of it.

The proof-of-concept attack on Google Gemini demonstrates how the damage is magnified when the LLM is directly connected to other sources of data. In this case it leverages the LLM’s access to Google Workspace  to quickly and neatly plunder emails, messages, documents and contact lists.

The prompt injection attacks work with an external LLM, but pose an even greater threat when interfacing with internal LLMs that were otherwise believed to be secure from malicious outside contact. These model types not only often contain more sensitive company and personal information, but are also less restricted in terms of safety guardrails that might otherwise slow down or disrupt an attacker’s pattern of prompts.

The researchers note that this is a case of something being more of a feature than a bug; numerous legitimate extensions already hosted on the Chrome Web Store leverage the direct browser access to provide various tools and features. Users can resist this attack type by being extremely vigilant about what browser extensions they install, but they have little control over a trusted existing extension being hacked. Hackers might also slip in via other means, such as phishing, and install an “invisible” extension the user is not aware of.

Prompt injection attacks can easily evade detection

Ease of infiltration and access is a major concern, but at least equally worrying is how hard these prompt injection attacks can be to detect once they are up and running.

The malicious extension does not need to acquire any new permissions because the browser already has scripting access (via the Document Object Model permissions) sufficient to let it enter commands directly into an LLM without any user interaction. Since this is a “legitimate” and approved action in the eyes of the system, security software is very unlikely to detect the malicious extension operating in the background unless it happens to be identified and blacklisted.

The researchers note that there are some ways to counteract these attacks, however, chiefly by monitoring DOM interactions within any LLMs or tools that are built on LLMs. Most of the defense is screening and prevention, however; regularly review extensions to ensure they are safe and still necessary, and screen new installs with behavioral risk assessment tools.