Anthropic Warns Small AI Models Could Soon Become Self-Replicating Worms Targeting Vulnerable IoT Devices

As the world worries about the threat of the biggest and best frontier AI models, Anthropic is warning about a smaller threat that could go under the radar: more modest models that could fit onto poorly-defended IoT devices and replicate themselves without the assistance of a central data center.

The threat model is basically that of the typical computer worm that has existed for half a century, but with an AI intelligence providing it with an added boost. Anthropic thinks the threat is not yet developed enough to be a major concern, but could be as new updates to AI models are released throughout 2027.

Researchers warn small AI models could target IoT devices that aren’t patched

IoT security has been lambasted since these devices began to appear, and it has not improved tremendously since: default administrator login credentials, a total lack of security patches, and so on remain common complaints as manufacturers continue to “race to the bottom” to compete on price.

Smaller AI models are poised to make a near-term jump in capability that would allow them to self-replicate via these vulnerable devices, needing only 4 GB of RAM to use one as a viable node. And though they would benefit from a central “command and control” hosted on a server somewhere, they do not absolutely need one to replicate to new devices. While frontier AI models running from data centers could be controlled by a “kill switch,” something that the US and other governments are now considering as a regulatory requirement, one could not centrally deactivate a fleet of these independent AI worms in the same way.

At least in the near term, the small AI models will not be capable enough to engage in social engineering approaches (as some of the frontier models did during the attacks recorded this year). However, they are already more than capable of probing for known vulnerabilities and exploiting them, as well as combing GitHub and other sources for unintentionally exposed credentials.

And while the baseline in Anthropic’s testing for these devices was a cheap 4 GB Android smart TV, the researchers believe small AI models could split into “shards” that work together to more effectively exploit target devices with modest hardware.

Cheap “off brand” devices most vulnerable

The experiment that Anthropic did was something of a smaller-scale replication of the infamous Kimwolf botnet, which racked up about two million IoT devices while primarily targeting a similar type of cheaper-end Android TV. Numerous of these are pumped out by China-based manufacturing with a fork of the Android Open Source Project (AOSP) as their operating system, no additional security, and absolutely no patching to be expected once in the hands of the consumer.

The lynchpin of this threat is the expected capability of current AI models that are small enough to run on such a smart TV with 4 GB of RAM without tipping the user off by interrupting normal function. Right this minute, the researchers say the most capable models are not quite there yet. However, updates coming within “months” are expected to push them over the edge into making this approach viable in the wild. However, even when this self-replication becomes possible it will likely be slow (about one to two days for one of these AI worms to replicate to a new device).

What makes the threat more significant is that the AI models will be able to learn as they go, compromising new types of devices rather than executing a simple script, and leave notes about what they have discovered for the next generation of replicants. But they will be slow, and will struggle with advanced capability without having a data center to call back to. The biggest risk is the fact that many organizations now have sprawling inventories of IoT devices that may not only have poor security, but that they have lost all visibility of entirely.