Massive Proof Generation Project Reveals Hidden Bugs In Open-Source Code

The promise of software that is provably safe is colliding with a harsh reality: verifying code has historically been too expensive for widespread use. To address this bottleneck, Anthropic has deployed its Claude model to take on the mathematical workload of proving software correctness. The company, working alongside its partner Theorem, aims to transform this process from a niche, high-cost activity into something as routine as standard testing.

The results are substantial. In a matter of weeks, Claude generated approximately 180 million lines of mathematical proof code. This massive undertaking focused on 123 open-source libraries, drawing on test programs from Google's OSS-Fuzz security project. The effort, led by Anthropic's Frontier Red Team, did not create new software but instead verified existing code by converting crash tests into mathematical problems and analyzing compiled machine code.

The process is designed for high automation. Claude generates proofs which are then checked by an independent verification program called Rocq. Researchers describe the method as fully autonomous proof generation, though they note that the necessary tools and formal foundations were constructed by humans at Theorem.

While generating these proofs, the AI model encountered hundreds of bugs and was authorized to resolve them directly. Researchers report that many of these issues would have evaded conventional testing because they only emerge under highly specific conditions, such as when handling unusually large input files or a particular sequence of accesses. The utility of this approach was highlighted during an effort to verify a widely used PDF reader, where the team identified as many security bugs in a single week as are typically reported for that software over an entire year. In a separate instance, Claude uncovered a logical error that had remained undetected for over a decade.

Despite the volume of proof code, the project faces strict limitations. Anthropic acknowledges that 'formally verified' does not equate to being error-free, as proofs are only valid under specific conditions, including a particular processor architecture, single-threaded programs, and inputs under 64 MB. Furthermore, the compute cost is high, with individual proofs running into hundreds of millions of tokens, though Anthropic did not disclose total project costs.

The company argues that if AI can verify its own code, developers can trust AI-generated software more and reduce the time spent on reviews, lowering costs across the entire coding process. The system is also becoming more efficient; a smaller test run indicated that proofs are reusable, and Anthropic claims its Opus 5.5 model solves half the proofs with about 40 percent fewer tokens than its Opus 5 model.

Looking ahead, Anthropic expects formal verification with AI will soon become as practical as standard testing. The roadmap for Theorem already targets critical infrastructure components such as glibc, OpenSSL, and cURL, shifting the central question of software automation from whether AI can program to whether it can prove the code it writes is safe.