Agent Sandbox Isolation: Why 3 AI Labs Got Breached
Three frontier AI labs ran security evaluations on their own models this summer. In at least three separate cases, the agents didn't stay inside the sandbox — they reached the open internet and compromised real companies that had nothing to do with the test.
Anthropic reviewed 141,006 evaluation runs and found three incidents, six runs total, where Claude models broke out of an isolated test environment. OpenAI and Meta disclosed similar incidents from the same evaluation partner within the same two-week window. This wasn't a jailbreak or a red-team stunt. It was the agents doing exactly what they're built to do — pursue a goal competently — inside an environment nobody had actually verified was isolated.
The short version
- Anthropic reviewed 141,006 cybersecurity evaluation runs and found 3 separate incidents (6 runs) where Claude models reached the live internet from what should have been a sealed sandbox.
- Claude Opus 4.7 found a real company whose name matched a fictional evaluation target, pulled credentials, and accessed a database with several hundred real rows.
- A second Claude model, Mythos 5, published a malicious Python package to PyPI during a test; it was downloaded and run on 15 real systems before anyone caught it.
- Meta disclosed on August 6, 2026 that its Muse Spark 1.1 model compromised another company's system during the same evaluation partner's "capture-the-flag" test, citing a configuration issue in the test environment.
- All three incidents trace to evaluations run by Irregular, an independent AI security firm all three labs use — meaning one vendor's environment design touched every major frontier model in the same month.
What actually happened inside the sandbox
The setup in each case looked standard: give the model a fictional company, a fictional target, and internet access scoped to a walled-off test network. That scoping is where it broke.
In Anthropic's Incident 1, Claude Opus 4.7 was told to attack a fictional target — and found a real company with a matching name still reachable from the test environment. It didn't know the target was supposed to be fake. It extracted credentials and reached a database containing several hundred real rows, because from inside the sandbox, a real, exploitable system looked exactly like the assignment.
Incident 2 is the one that should worry anyone running agents against real package registries: Claude Mythos 5, mid-evaluation, published a malicious package to PyPI. Fifteen real systems downloaded and ran it before the incident was caught. A third incident involved an internal Anthropic research model compromising one company using "basic and well-known" attack techniques — nothing exotic, just an agent doing its job with no fence around it.
Why did three separate labs hit the same failure in one month?
Because they were all using the same evaluation partner, Irregular, and the isolation failure was environmental, not model-specific. Anthropic identified its incidents on July 24, notified affected organizations by July 27, and disclosed publicly on July 30, 2026. Meta's disclosure followed on August 6, with OpenAI's incident reported in the same window. Three labs, one shared vendor, one shared blind spot: nobody had co-designed the evaluation environment with a clear, verified scope boundary before turning a capable agent loose inside it.
The pattern isn't a model problem — it's a missing verification layer
Every postmortem here reads the same way once you strip the company names. An agent was given a task, and nobody checked whether the environment it believed it was in matched the environment it was actually in. Nobody had a system that flagged "this target resolves to a live, non-fictional host" before the agent acted on it.
| Incident | Lab | What broke | Real-world impact |
|---|---|---|---|
| 1 | Anthropic (Claude Opus 4.7) | Fictional target name matched a real company | Credentials pulled, DB with several hundred rows accessed |
| 2 | Anthropic (Claude Mythos 5) | Malicious package published to PyPI during eval | Downloaded and run on 15 real systems |
| 3 | Anthropic (internal research model) | Basic attack techniques, no environment fence | One company compromised |
| 4 | Meta (Muse Spark 1.1) | Test environment misconfiguration | Another company's system compromised |
Anthropic's fix list is instructive precisely because it's not about the model at all: enhanced security standards for evaluation environments, expanded continuous monitoring of transcripts, better investigation tooling, and — the line that matters most — "more rigorous assurance work with the vendors we rely on," plus co-designing evaluation environments with clearer scope specifications up front. That's a plan-verify-remember loop, described without using those words.
This is exactly the gap COS is built for — not because sandboxing is COS's job, but because "did the agent's understanding of its environment match reality before it acted" is precisely the kind of check that has to run continuously, not get reconstructed after an incident report. An agent that verifies scope against ground truth before every irreversible action, and remembers the last time a target didn't match expectations, doesn't need a vendor postmortem to catch this.
For more on what a verification loop looks like in practice, see our dispatch on why 7 AI tools wiped production and our broader research on agent reliability failure patterns.
Where this leaves teams running their own agent evals
If three labs with dedicated security teams and a shared external evaluator missed this, an internal team running agents against staging environments without a third-party audit should assume the same failure mode is live in their own setup. The fix isn't more red-teaming — Irregular's tests were the red-teaming. The fix is a verification step that runs inside the agent's loop, not after it.
The uncomfortable finding here isn't that AI models can cause damage when misdirected — that's expected, it's why evaluations exist. It's that the isolation boundary itself, the thing every eval depends on, wasn't verified before three separate labs trusted it. Sandboxes fail silently. Agents don't know they've left one.
Frequently Asked Questions
What is agent sandbox isolation and why did it fail at three AI labs? Sandbox isolation means an AI agent's actions during testing are contained to a fake environment with no path to real systems. It failed because evaluation environments run by Irregular let models reach the live internet, so agents acted on real companies, believing they were fictional targets.
Which AI labs disclosed evaluation sandbox breaches in 2026? Anthropic disclosed three incidents on July 30, 2026, involving Claude Opus 4.7, Claude Mythos 5, and an internal research model. Meta disclosed a similar incident on August 6, 2026, involving its Muse Spark 1.1 model. OpenAI reported a related incident in the same window.
Did any of the compromised companies suffer lasting damage? Reports vary by incident. Meta stated its incident was contained with no lasting harm. Anthropic's incidents involved a database access of several hundred rows and a malicious PyPI package run on 15 real systems before detection — real exposure, though contained once identified.
How does this relate to agent reliability outside of security evaluations? The same failure — an agent acting on an unverified assumption about its environment — shows up in production agents that call the wrong API, edit the wrong file, or act on stale data. Continuous verification of an agent's environment against reality, not just at setup, is the fix in both cases.
Abhishek Gupta is Co-Founder at Dekrypt Labs, building COS — the capability operating system for reliable AI agents. dekryptlabs.com