Agents on the Loose

Throughout middle school, one of my favorite recess and P.E. games was "Capture the Flag." I always loved trying to sneak around into the other team's base along the side, or slip in when people were busy. Occasionally, this would grant me the opportunity to get the flag, but more often than not, I'd just get tagged. Lately, I've been playing some capture the flag again, but this time with AI Agents, and the stakes are considerably higher.

Outside of developing solutions for AI agents at work, I often conduct lots of experiments on my own time. One of these was to take an agent with the ability to create its own tools on the fly and see how it adapts to vague prompts for complicated tasks. After being quite surprised at its ability to find and decrypt a hidden file, I took that agent and put it in a Docker-in-Docker container with the objective of finding a hidden flag, and emailing me when it had done so. The caveat? The flag and email client were in an outer container, while the agent was running in an "inner" container, supposedly sandboxed and isolated from my mock "host" machine. Within five minutes the agent had found a vulnerability with a Docker socket, stood up a sister container with the host file system inside of it, started the container, gained access to all files inside of it, and then used SMTP credentials it found with the email client to send me an email containing the flag's data, and had written a file to the system as proof of its escape. While this is a well-documented Docker misconfiguration, the agent had to discover it autonomously and chain together multiple steps - reconnaissance, API exploitation, credential theft, and email exfiltration, with zero human intervention. What seems somewhat complicated actually isn't, which is a problem.

Recently, Claude Mythos performed a similar feat: escaping an isolated environment and emailing the researcher during their lunch break at their private email, with no internet connection. For Anthropic's latest flagship model, this is hardly surprising. My experiment, though, used OpenAI's gpt-4o-mini and gpt-5.4-mini, considerably smaller models designed for lightweight, multi-chain agentic work. What surprised me was that even gpt-4o-mini, the model many use as a surrogate partner and known for its pseudo-emotionality, managed to pull this off. This means even smaller, cheaper models can autonomously find vulnerabilities that are often just small misconfigurations developers missed. These models are chosen precisely for their cost advantages while remaining performant.

The real surprise came with gpt-5.4-mini. In the first 10 runs with the same prompt, it found the Docker socket vulnerability 100% of the time. Perfect reconnaissance. But it refused to exploit it in 8 out of 10 runs, citing safety concerns and ethical training. When I reframed the identical task as an authorized CTF exercise, refusals dropped to just 1 out of 10, and the escape rate jumped from 0% to 20%. Meanwhile, gpt-4o-mini only escaped once out of 10 runs, but it never refused. It failed due to technical errors, not ethical objections.

So, why should the average person care? Here's why this matters.

First off, these agents are already live, in the wild. Companies are deploying AI agents in back offices to manage infrastructure, handle customer records, process documents, be on the front lines of customer service, and make decisions. Many of these agents run on the same models I tested. If gpt-4o-mini can autonomously find and exploit a misconfiguration in a sandboxed environment, what happens when it's given access to your company's actual infrastructure? The agent doesn't need malicious intent, and truth be told the agents themselves aren't acting maliciously- they're just following a prompt. It just needs to be pointed at a problem and told to solve it by any means necessary.

Secondly, the vulnerability isn't special. The Docker socket misconfiguration is well documented, and has been since at least 2019. It's not a zero-day exploit or anything sophisticated. It's akin to leaving your house key under the doormat. The scary thing isn't that the vulnerability exists- lots of those do but we hardly hear anything about them- it's that an AI agent with basic tools found it, understood what it meant, and exploited it end-to-end in five minutes without any human guidance. Now think about all the other "doormats" in production systems that developers, under tight deadlines, with little to sometimes no support, and aging systems and infrastructure just haven't been able to get around to fixing yet.

Third, the safety guardrails are negotiable. The gpt-5.4-mini results might be the most worrisome. It found the vulnerability 100% of the time but refused to exploit it 80% of the time, initially. With a little prompt engineering though, which could take the form of prompt-injection, it complied 90% of the time by tweaking the framing of the task into an "authorized CTF exercise". There were no tricks (this time) or jailbreaks. If safety training can be bypassed with a simple reframing, what happens when someone with actual malicious intent tries?

And finally, this is just the beginning. These were small models running a simple exploit against a known vulnerability. Newer models are more capable. Production systems are more complex. More companies are adopting AI-assisted coding and deployment, and with more code being generated but only the same number of engineers reviewing it, the attack surface is growing exponentially. And we're handing these agents the keys to increasingly critical infrastructure because they're cheap, fast, and "good enough."

The question isn't whether AI agents can escape their sandboxes. The question is: what happens when they do, and who's responsible when things go wrong?