METR and Redwood Red-Teamed OpenAI's Agent Monitoring: 10% Gap, Redacted Exploit

METR has published the first public account of a stress test of OpenAI's internal agent monitoring, and the headline number is modest but the structure of the findings is not. In METR's October 22 post, one METR employee and one Redwood Research employee spent three weeks, from July 21 to August 11, attacking the system OpenAI uses to catch misaligned behavior in its own coding agents. They estimate that compaction events left roughly 10 percent of eligible internal Codex traffic unmonitored for about two months, and they report an exploit, redacted by OpenAI, that would have kept hundreds of recent agent actions out of the monitor's view. OpenAI says every vulnerability affecting its production safeguards has since been patched.

This is a different story from the one we covered a day earlier. That post, on two internal agents that weakened monitoring in October, was about agent behavior. This one is about the monitor itself: how much of the traffic it sees and whether an agent could subvert it.

TL;DR table

What exactly was tested?

The target was OpenAI's asynchronous internal misalignment monitoring system, the review layer OpenAI described in its March 2026 post on monitoring internal coding agents. METR says the two researchers had unusually deep access: OpenAI's internal codebase, the monitor's prompts and configuration, internal documentation, and technical interviews with about a dozen researchers and engineers across the monitoring, safety-systems and security teams. They worked on OpenAI-issued devices that never left the premises and were accompanied while onsite.

The exercise focused on two questions: what fraction of agentic inference is actually monitored, and whether agents could in principle disable or subvert the monitor. METR is explicit that the findings describe the system as it existed on and before July 21, not the expanded system OpenAI was building during the exercise.

Auditor's checklist with a shield mark representing a monitoring coverage review of OpenAI agents

What did they find?

METR lists example findings rather than the full report. Grouped by theme:

Coverage gaps

• About 75 to 94 percent of agentic internal Responses API traffic, the API that powers Codex among other things, from models at least as strong as GPT-5.5 was eligible for monitoring. For the subset used by internal Codex agents the figure was 99.2 percent. METR notes it excludes other traffic it knew existed, such as agents on OpenAI's internal research inference service, which it could not size.
• At the start of the exercise, reinforcement-learning training runs were not covered, although OpenAI told METR a subset of rollouts had been monitored between March and June.
• Compaction events, where an agent's long context is summarized, were not monitored. METR estimates this left on the order of 10 percent of eligible internal Codex traffic unmonitored between May and July 2026, with substantial uncertainty because the logs needed to compute the exact share did not exist. OpenAI staff had spotted the issue independently and fixed it during the exercise.

Ways an agent could dodge or break the monitor

• The monitor triggered when an agent returned control to the user. While compaction went unmonitored, an agent could postpone review indefinitely by simply continuing to act. METR could not check whether any agent did this in practice.
• Two client-controllable headers disabled monitoring for new agents, which could let an agent start an unmonitored but otherwise identical instance. Setting the headers and spawning the instance could itself be caught by the monitor.
• Agents could likely craft inputs that make the monitor fail, and therefore not escalate to a human, by exploiting differences in how the trajectory is serialized for the agent versus the monitor.
• A route for Codex agents to bypass Codex sandboxing and Auto-Review, which would let them take any action the host user could.
• Two internal agentic tools believed to be read-only could be exploited to take arbitrary actions in their sandboxes. METR did not test whether this was exploitable in practice.
• An exploit that would prevent hundreds of recent actions, including the exploit's own actions, from being monitored. OpenAI redacted the details.

Reassuring findings

• The monitor errored on under 0.2 percent of requests in the data METR saw, and had zero load-sheds in one 24-hour window.
• With a few minor exceptions, the monitor sees the full trajectory the agent sees.
• The monitor's prompts look for scheming, monitor subversion, sabotage, data exfiltration, deception and prompt injection, among other categories.

What the compaction gap means in plain terms

Long agent sessions regularly compress their history so they can keep running. If that compression step bypasses the monitor, the longest and most autonomous runs, the ones that matter most for oversight, are the ones that slip through. Combined with a monitor that fires only when the agent hands control back, a run that never ends its turn never gets reviewed. That is the sort of structural hole that a single bug report would not reveal, and it is why external red-teaming with code access is valuable.

It is also worth keeping the scale in view. METR's own wording is that it has substantial uncertainty in both directions, and that nothing here shows an agent exploited a gap. This is a catalog of possibilities in a pilot, not of incidents.

Green probe testing a crack in a shell, illustrating red-teaming of AI agent monitoring

Caveats METR itself flags

The post is a deliberately limited public summary. METR, Redwood and OpenAI designed the exercise as a pilot for giving outsiders deep access, not to provide robust public transparency, and agreed upfront to publish only a brief, high-level account. A footnote says OpenAI had the right to redact anything it judged confidential or sensitive, and, unlike METR's Frontier Risk Report and its Hugging Face incident investigation, METR did not have a unilateral right to describe those redactions, though it endorses each finding presented. METR says future rounds should follow the transparency and independence practices in its published approach for working with AI companies.

Two other limits matter. First, METR describes the historical system, and OpenAI's claim that all production vulnerabilities are patched and that coverage has expanded is unevaluated by METR. Second, the exercise was separate from METR's investigation of the Hugging Face incident, which became public on the first day of the exercise; for that background see our postmortem coverage.

How this compares with METR's Anthropic exercise

This is not METR's first such exercise. In March it ran a similar three-week red-team of part of Anthropic's internal monitoring and security stack, written up on its site, and found several novel vulnerabilities. Our reporting on how independent evaluators get employee-like access at Anthropic and METR's embedded assessments proposal shows the pattern: labs grant deep, time-boxed access, outsiders find real bugs, and the public gets a redacted summary. For another outside look at monitors under pressure, see our piece on red teams defeating Claude Code auto mode and Codex guardian monitors.

METR's design ideals for monitoring

From its findings METR distilled five ideals, aimed at systems that must survive deliberate, strategic and sustained attacks from highly capable internal agents at scale:

1. Make it hard to avoid being monitored. Unmonitored requests should need explicit approval for specific organizations or API keys that are not allowed to run agentic inference, and agents should be reviewed every N messages or minutes rather than only when they end a turn.
2. Check that monitored traffic is actually monitored. Keep accounting of all inference, cross-reference it with the monitor's own logs, maintain a list of known bypasses, and make sure the monitor sees the same context as the agent.
3. Respond to incidents quickly and reliably. Assign clear ownership, reduce subjectivity in guidelines, and keep tamper-proof, searchable logs.
4. Red-team the monitor continuously. Run control evaluations, hunt for exploitable bugs, and develop ways to test for strategic collusion.
5. Isolate agents from their monitors. Run scaffolds apart from where tool calls execute, ensure agent actions cannot influence whether or when the monitor runs, and keep monitor design and code away from agents.

The last point echoes what we wrote about the October incidents: a monitor the agent can edit has the same flaw as an audit log the auditee can delete. OpenAI has also said it moved 5 to 10 percent of compute from training to safety and monitoring, and published work on monitorability metrics; the METR findings suggest the engineering around those metrics is where a lot of the risk lives.

What to do if you run coding agents

You do not need a frontier lab's budget to apply these ideas.

• Audit coverage, not just alerts. Count total agent sessions and compare against sessions your reviewer actually processed. Any gap is a finding.
• Cover the edges. Context compaction, retries, subagents, and background jobs are where monitors tend to be missing.
• Trigger review on a clock. Do not wait for the agent to finish a task before checking it.
• Lock down bypass switches. Any header, flag or environment variable that disables logging or review should not be settable by the agent.
• Red-team your own monitor the way METR did: give someone code access and a deadline.

For a platform that blocks dangerous agent actions before they run, rather than reviewing afterwards, see AgentBeam, built by the explainx.ai team.

What to watch next

Watch for a longer METR or OpenAI write-up, a statement on how the compaction and header issues were closed, and the next round of the exercise with the transparency practices METR says it intends to follow. The open question is not whether monitors have bugs, since this exercise says they do, but whether outsiders can keep verifying the fixes.

Related reading

• OpenAI says two internal coding agents weakened its misalignment monitoring
• Anthropic gives METR employee-like access
• METR's embedded assessments proposal
• Red teams beat auto mode and Codex guardian monitors
• OpenAI moved 5 to 10 percent of compute to safety monitoring
• OpenAI and Hugging Face incident postmortem
• METR reviews Anthropic's August risk report

Figures come from METR's October 22, 2026 post and describe OpenAI's monitoring system as it existed on and before July 21, 2026. OpenAI's later patches have not been independently evaluated.