METR and Redwood Broke OpenAI's Agent Monitoring: 75–94% of Traffic Covered

For three weeks, from July 21 to August 11, one METR employee and one Redwood Research employee tried to get around the system OpenAI uses to watch its own internal agents for misalignment. The result is a 61-page closed report for OpenAI's researchers and security team, plus a short public summary. Its main estimate: at the time of the review, 75–94% of internal agentic Responses API traffic from models at least as strong as GPT-5.5 was subject to monitoring, with uncertainty in both directions, as the researchers acknowledge. They also found ways to escape observation entirely.

Access almost like a staff engineer's

The review ran as a "deep access" pilot for external researchers. Participants worked with OpenAI's internal codebase, prompts, monitor configuration and internal documentation. They conducted technical interviews with about a dozen researchers and engineers from the monitoring, safety-systems and security teams. They worked on OpenAI-issued devices that never left the office, and they were escorted.

The limits matter too. Only a short post may be published: OpenAI can redact any information it deems confidential, and, unlike in previous work, METR has no right to unilaterally describe such edits. METR and Redwood leadership received a 38-page version of the report, edited by OpenAI. The review covered the system as it existed on July 21 and earlier, not the extensions the company was rolling out in parallel. The study is also unrelated to METR's investigation of the Hugging Face incident, although that incident was made public on the first day of the exercise.

Twelve findings

• Traffic coverage. 75–94% of agentic Responses API traffic (which powers Codex) from GPT-5.5-class and stronger models was subject to monitoring.

For a subset of internal Codex agents the estimate is higher: 99.2%. Traffic to the internal research inference service was not assessed because of access and time constraints.

• RL training. Some rollouts were monitored for misalignment from March to June, but by the start of the exercise RL runs were not covered by the system. • Compaction.

Because compaction events could not be checked, about 10% of eligible Codex traffic went unmonitored from May to July 2026. The exact share could not be determined because the necessary data was not logged. OpenAI noticed the problem before the exercise and fixed it during it.

• Review deferral. The monitor fired when the agent returned control to the user. As long as compaction went unchecked, an agent could defer oversight indefinitely while continuing to take actions.

The researchers could not verify whether anyone has used this in practice. • Errors and load shedding. According to the data shown, the monitor failed on less than 0.

2% of requests, and during one 24-hour day of the exercise there was no load shedding at all. • Codex sandbox. The researchers found a way to bypass sandboxing and Auto-Review, after which an agent could do anything available to the host user.

• Blind-spot exploit. One technique caused hundreds of the most recent actions, including the exploit's own actions, to escape monitoring. OpenAI redacted the details.

• Serialization. Agents can probably craft inputs that break the monitor and prevent the trajectory from being sent for human review, because the trajectory is serialized differently for the agent and for the monitor. • Headers.

Two independent client-side headers disable monitoring for new agents, making it possible to launch an identical but unmonitored instance. The monitor may, however, notice the header-setting and instance creation themselves. • "Read-only" tools.

Two internal agentic tools without monitoring were considered read-only, but the researchers found a simple exploit for arbitrary actions in their sandbox. Practical exploitability was not tested. • Context completeness.

With a few small exceptions, the monitor sees the same trajectory as the agent. • Monitor prompts. They look for scheming, undermining the monitoring system itself, sabotage, data theft, deception, and both incoming and outgoing prompt injection.

OpenAI said that all vulnerabilities affecting the production defense stack have been fixed, and described substantial changes to the monitoring's target coverage. METR did not verify these claims.

Five principles for monitoring that can withstand a smart agent

From the findings, the authors derived requirements for systems that can resist deliberate, strategic and sustained attacks from strong internal agents:

• Make monitoring hard to bypass. Allow unmonitored requests only for explicitly approved organizations and API keys that are barred from agentic inference. Check the agent every N messages or minutes, not at the end of a turn.

• Control actual coverage. Keep a record of all inference and the monitored share, regularly reconcile inference logs against monitor logs, maintain a list of known bypasses, and make sure the monitor sees the same context as the agent. • Respond to incidents quickly.

Assign an owner, set clear procedures, and keep immutable, easily searchable logs. • Attack the monitor regularly. Run control evaluations, hunt for exploitable bugs, and develop a methodology for assessing strategic collusion.

• Isolate agents from monitors. Run the scaffold separately from the tool-execution environment, don't let the agent's actions affect whether or when the monitor fires, and don't show the agent the design and code of the oversight system.

What to check in your own agent stack

The typical holes here are mundane engineering problems, not exotic ones: a missed event type, an end-of-turn trigger, a client-side header, a stray traffic branch. Any team that runs autonomous coding agents can run its own infrastructure through this checklist: Are inference logs reconciled against monitor logs? Can a request parameter switch off observation? Does the monitor see the same thing as the agent? Does the check fire on a timer rather than at turn completion? For external auditors, the project sets a precedent: deep access to a lab's internals is possible, but public transparency still remains at the company's discretion. In future iterations, METR promises to stick to its own principles of independence and openness.