Check Your AI Workflows: Researchers Find Simple Means of “Identity Hijacking”

A new report from Noma Labs is an eye-opener for organizations rapidly onboarding AI workflows. An attacker may well be able to extract internal sensitive information from public inputs like a support email address or chat agent, no hacking required.

The issue, dubbed “Workflow Identity Hijacking,” targets vulnerable transitions between workflow steps. But to say it “targets” sounds like giving the method too much credit as a form of attack; oversights in the handling of permissions make it possible for the attacker to literally just ask the agent for whatever private information they are looking for.

Can AI workflows just be asked to give up internal information?

The researchers did not delve into all the specific AI workflows impacted by this flaw, but it seems to be an endemic issue throughout the industry. They did name Google Workflows as one that was impacted prior to the report’s publication (since updated to address the issue).

The central problem is that AI workflows are failing to address authentication checks at transitions between workflow steps. Too often, the AI seems to assume the requester has the same advanced permissions it has. When an AI input faces the public internet, this means that anyone can potentially come along and take advantage. In the example given by the researchers, there is no hacking of the prompt involved; an email support AI is simply asked for private information as if it was excluded from documents that the attackers had already received.

The “attack” is simple: the attacker asks the AI email system or chatbot for some sensitive internal business information, like sales figures or meeting notes, and the AI spits it back. In the transition between workflow steps, the AI does not recognize that what it is being asked for may require a level of access the requester does not have. Organizations can address this through various means, but this can involve settings and policy that are not always common or seen as necessary.

Securing AI workflows falls to developers, but organizations can implement protection layers

The researchers note that this is not a prompt injection attack, and should not be treated like one. The attackers have no need of trickery. It is a simple case of decoupling identity and permissions between AI workflows and the requester, and fundamentally it is something the AI developer needs to address.

That does not mean there are not defensive options available to organizations. The researchers suggest a number of measures such as identity-aware token delegation to replace static administrative API keys, and asymmetric output separation to keep privileged AI workflows and their permissions siloed from general public requesters. And if they are not already, IT teams need to begin regarding AI workflows as being on par with human users that have the same privileges and access. That fundamentally means improved identity governance, with both requesters and AI performers of tasks properly monitored and logged whenever broad public access to an input is possible.