US Secretary of State Impersonated; State Officials Targeted as AI Deepfakes Advance in Capability

Not a long time ago, only a matter of mere months, AI deepfakes were only a modest threat at best. They could be somewhat convincing, but were usually easily spotted and only racked up a relatively small number of serious breaches and successful scams. But generative LLMs and associated tools are rapidly improving, and threat actors are becoming more bold about deploying them against high-level targets.

The latest incident involves fakes made of US Secretary of State Marco Rubio, who was just the victim of another deepfake-driven propaganda campaign earlier in the year. An inside source has told reporters with the Washington Post, Reuters and other media outlets that another attempt involving his voice and writing style was made in an attempt to compromise assorted US and foreign officials, and a warning has gone out about the increasing potential of AI deepfakes and the need for awareness of how realistic they can now be.

Potential of AI deepfakes is improving with recent LLM updates

It should be noted that the inside sources indicate the Rubio AI deepfakes were not regarded as overly sophisticated or much of a threat. The attackers seem to have limited themselves to cloning his voice, used to send several voicemail messages, and using a generative AI to replicate his writing style to attempt to entice targets via text message and into Signal chats. But generative AI tools have seen some major leaps in recent months that can make deepfakes much more realistic in the hands of a capable user.

The attacks involving Rubio thus far do not appear to have been the work of capable users, limiting the damage. The first incident took place in March of this year, when an unknown source began circulating videos on social media that purported to be of Rubio making anti-Ukraine statements. This turned out to be an AI voice overlay of existing video of Rubio’s interactions with Ukranian president Zelensky. An American political consultant working for a long-shot Democrat primary rival also created AI deepfakes of Joe Biden’s voice for robocalls in early 2024, in what appeared to be an attempt at voter suppression in New Hampshire.

AI deepfakes have already proven to be effective in the hands of criminals, however, and the biggest incident thus far took place over a year ago when the technology was much more limited. In January 2024, major international design firm Arup was hit by a hacker who stole $25 million dollars by tricking a payroll department employee using a combination of old video conferencing clips and a deepfaked clone of the CFO’s voice.

Though there have not yet been similar high-profile breaches tied to AI deepfakes, given the current state of tools like ChatGPT’s GPT-4o and Google’s Deepmind it would seem to be only a matter of time until this segment of crime erupts (and probably not too far into the future).

Rubio AI deepfakes “unsophisticated,” but attackers had some knowledge of State Department internal procedures

There is not yet any word of who the specific targets of the AI deepfakes were or what the attackers were after, but these were likely attempts to obtain login credentials. The attacker targeted several foreign officials and members of the State Department via assorted means, but in all cases used either “vishing” or “smishing” approaches.

Though their AI game may not have been at the top of the charts, the attackers did appear to have some inside information about State Department procedures that is not necessarily available to the general public. They made use of technology logos, branding, naming conventions and internal documentation in their phishing messages, pointing to a possible state-backed actor working from prior espionage knowledge.

At least in terms of technological capability, the attackers are likely ahead of the defenders in terms of AI deepfakes at this point. It is just a matter of them refining and properly applying their technique. These attacks will also not be limited to public figures; everyday people can expect AI scam attempts as tools become cheap and accessible enough. Technical measures, such as those already being implemented by major social media platforms, can help; but at the end of the day, avoiding these scams will require good old-fashioned awareness of attacker capability and whether or not an incoming communication can be traced back to a legitimate source.