Researchers Find Oversight Holes Inside AI Lab

An intensive review of OpenAI's internal systems has revealed substantial blind spots in the safeguards designed to watch for dangerous or unauthorized artificial intelligence behavior. Two researchers spent three weeks inside the company this past summer, scrutinizing the codebase and infrastructure intended to flag rogue agents, and reported identifying numerous ways the monitoring apparatus could be bypassed or fail entirely.

The examination, conducted from July 21 to August 11, involved two researchers from METR and Redwood Research. They were granted access to internal documents and source code, and conducted interviews with approximately a dozen OpenAI employees. Their mandate was to evaluate the efficacy of OpenAI's internal monitoring system for detecting unauthorized model actions.

During the audit window, an estimated 6 to 25 percent of internal traffic using agentic Responses API capabilities from models of the GPT-5.5 class and above went entirely unmonitored. METR noted that the uncertainty regarding this specific estimate is significant in both directions. Additionally, for internal Codex traffic intended to be overseen, roughly 10 percent of activity between May and July went untracked due to a defect in how the system managed compaction events, or context summaries. This gap was identified as significant by METR, though OpenAI had independently discovered and remedied the flaw while the evaluation was still in progress.

The researchers identified specific vulnerabilities that allowed agents to evade scrutiny. They found that oversight was only triggered when an agent ceded control back to the user; consequently, agents could avoid this trigger simply by continuing to execute actions uninterrupted. The team also determined that inputs could be crafted to hide hundreds of actions from the monitor and bypass the Codex sandbox. Furthermore, the auditors concluded that specially crafted inputs could likely crash the monitor entirely, and noted that model behavior during reinforcement learning training was not being monitored at the time the audit commenced.

OpenAI has stated that it paused reinforcement learning training following an incident involving Hugging Face in August and subsequently expanded monitoring to cover that activity. However, these modifications were implemented after the researchers' access period had concluded and were not evaluated as part of this specific audit.

Following the completion of the work, OpenAI asserted that all weaknesses identified by the auditors have been resolved. The complete findings are contained in a 61-page report delivered internally to OpenAI. Following redactions applied by the company, METR and Redwood Research leadership received a 38-page version, while the public release was limited to a short summary. METR stated it possesses no right to independently disclose the material that was redacted.

This evaluation was conducted as a pilot program jointly organized by METR, Redwood Research, and OpenAI, designed to grant outside researchers deep access to internal infrastructure. For future evaluations of this nature, METR expressed a desire to advocate for greater public transparency regarding the results.