🔍 Read the full analysis: The Promise And Limits Of Anthropic’s Claude Formal Verification Work on ThorstenMeyerAI.com
Get hardware and tech essentials delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Anthropic and collaborator Theorem report that Claude proved a defined no-crash property for compiled programs covering 239 fuzz-test entry points across 123 open-source libraries. The effort also found bugs, but the proofs apply only to specific RISC-V binaries and stated assumptions; they do not establish that the libraries are bug-free.
Anthropic’s Frontier Red Team and its collaborator Theorem reported October 22 that Claude proved a defined no-crash property for 239 fuzz-test entry points across 123 open-source libraries. The proofs apply to compiled RISC-V programs and inputs under 64 megabytes, under specified assumptions; the project also found bugs that the teams said existing fuzzing had missed.
The team used existing fuzz tests as the basis for each proof. A fuzz test supplies an entry point into a library, while the formal specification asks whether the compiled program can crash for any input within the stated size limit. The programs were compiled as RISC-V executables instrumented with AddressSanitizer, which causes certain memory errors to halt execution. Claude generated proofs in Rocq, a proof assistant, and Rocq’s checker verified them against the specifications.
Anthropic reports that the work ran over a few weeks and produced about 180 million lines of Rocq proof. A typical proof was about 600,000 lines and covered a binary with a median length of 18,000 RISC-V instructions. The company says the proofs did not require people to review every generated line: confidence instead depends on the theorem statements, the formal models of RISC-V and Linux processes, and the Rocq checker, along with other components in the trusted computing base.
According to Anthropic, the process discovered hundreds of bugs, including problems involving large or unusual inputs and sequences of calls that ordinary fuzzing may not reach. Examples in the source account include an out-of-bounds write triggered by a rare layout in an input of around 20 megabytes, and an image-decoder issue requiring a 64-megabyte input and more than 2 gigabytes of memory. The account also describes a file-loading restriction that was not always enforced, a logic flaw that did not necessarily cause a crash.
Infographic 01 · From test to proof
Existing fuzz tests became specifications. Claude proved them.
Over a few weeks, Claude proved at the machine-code level that hundreds of real fuzz entry points can no longer crash, after patching the bugs it found.
The pipeline
From an existing test to a checked proof
- Compile. The library and its existing fuzz test become a RISC-V executable, instrumented with AddressSanitizer.
- Specify. Theorem’s RISC-V and Linux process semantics plus the no-crash property mechanically yield the specification.
- Prove. Claude writes the proof in Rocq. A failed attempt points at a possible bug; Claude patches it and proves the new spec.
- Check. The Rocq proof checker verifies the proof. Virtually none of the proof needs human review.
Fuzzing vs. proof
Same entry point, different guarantee
| Fuzzing | Proof | |
|---|---|---|
| Inputs | Fuzzing: Generated samples, typically capped at 1 MB | Proof: Every input under 64 MB |
| Paths | Fuzzing: Guided by code coverage, no guarantee | Proof: Every path reachable from the entry point |
| Result | Fuzzing: Bugs found so far | Proof: No crash left to find on the modelled system |
Typical proof: about 600,000 lines for a binary with a median length of 18,000 RISC-V instructions.
“If Claude can prove this spec, the fuzz test on RISC-V becomes superfluous.”Anthropic, 22 Oct 2026
Source: Anthropic, “LLMs are making formal software verification feasible at a much larger scale” (22 Oct 2026).
Powered by Thorsten Meyer AI
What the Proofs Can Establish
The result points to a possible way to apply formal verification to more software: use a test developers already have as a specification, then let an AI system generate a proof checked by a separate verification tool. If that process can be made cheaper and more reliable, it could extend verification beyond projects with large dedicated teams. Anthropic argues that this matters as software vulnerabilities remain a concern even in heavily tested codebases.
The distinction between proof and testing is important. Fuzzing tries many inputs, but cannot by itself establish that every possible input was tested. A successful formal proof can establish the stated property for all inputs in the model’s scope. Here, that property is not general correctness: it is that a particular compiled program does not crash at a particular entry point for inputs under 64 megabytes, subject to the specification and assumptions. The work therefore offers a stronger guarantee than a finite set of test runs, but a narrow one.
Cost remains a practical constraint. The source account says some proofs can consume hundreds of millions of tokens, even as newer models and reuse of shared proofs may reduce the amount of work. The scale of the output also shifts attention from reading generated proof text to checking the machinery and assumptions that make the proof meaningful.
formal verification software tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
From Existing Fuzz Tests to Rocq
Formal verification has historically required substantial specialist effort. Anthropic’s account compares this project with the seL4 microkernel effort, citing two person-years to build the system and 20 person-years to verify it. That example helps explain why rigorous verification has been concentrated in a limited set of high-assurance applications rather than applied routinely across large software ecosystems.
The new method starts with a narrower target than proving a whole library correct. A fuzz test is compiled together with the library, and the resulting executable is modelled at the instruction and process level. Claude attempts to prove that the instrumented program cannot crash under the defined conditions. If a proof attempt fails, that failure can point to a potential defect; the system can then patch the code, regenerate the specification and attempt a new proof.
Anthropic describes the project as, to its knowledge, the first large-scale use of AI to apply formal verification to production software at assembly-code level. That is the company’s characterization, not an independently established ranking. Anthropic says it is publishing proofs that required no patching, previously unpublished trusted components, and specifications for fuzz tests it could not prove.
“The work is, “to our knowledge,” the first large-scale use of AI to apply formal verification to production software at the assembly-code level.”
— Anthropic
As an affiliate, we earn on qualifying purchases.
Where the Guarantees Stop
These proofs do not show that the libraries are bug-free. They cover selected fuzz-test entry points and a specific no-crash property, not every function, behavior or security condition. A program could satisfy that property while still producing incorrect results or having flaws that do not trigger a crash. The reported file-restriction logic bug illustrates that distinction.
The scope is also tied to the build and model: the source account specifies single-threaded static RISC-V binaries linked against musl. It does not establish the same result for the x86 or ARM builds linked against glibc that many users run. The conclusions depend on the accuracy of the models, theorem statements, compiler and other trusted components, as well as the assumptions used in the proof.
The source material does not state how many reported bugs have been independently confirmed, fixed or disclosed to maintainers, nor does it provide a complete accounting of token use or total project cost. It also notes that some proof attempts do not finish. Those details matter when judging how readily the approach can be adopted outside this reported set of test cases.
fuzz testing tools for open-source libraries
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Publication and Broader Testing
Anthropic says it will publish successful proofs that did not require code patches, trusted components that were not already public, and specifications for tests it could not prove. Those materials should give outside researchers a chance to examine the formal statements and the verification setup. The source account does not give a publication schedule or say when the remaining materials will be available.
Further evaluation will depend on whether independent reviewers can reproduce the proofs, assess the trusted components and confirm the reported bugs. It will also be useful to see whether the approach can cover more architectures, build configurations and behaviors beyond crash-related properties, and whether proof costs fall enough for routine use. Until then, the reported results establish a significant demonstration on a defined set of binaries, not a general guarantee about the security of the underlying libraries.
As an affiliate, we earn on qualifying purchases.
Key Questions
What did Claude prove?
Anthropic and Theorem report that Claude proved a no-crash property for 239 fuzz-test entry points in 123 open-source libraries. The property applies to specified compiled RISC-V programs and inputs under 64 megabytes, subject to the formal model and assumptions.
Does this mean the libraries are bug-free?
No. The proofs cover selected entry points and a particular crash-related property. They do not establish that every library behavior is correct or that every security flaw is absent.
How were the proofs checked?
Claude generated proofs in Rocq, and Rocq’s proof checker verified them against formal specifications. The result also relies on the accuracy of the specifications, models and other trusted components.
What bugs did the project find?
Anthropic says the work found hundreds of bugs. The examples described include memory errors triggered by unusually large or specifically structured inputs, a flaw requiring a sequence of calls, and a file-loading restriction that was not always enforced. The source account does not give a complete independent confirmation or remediation tally.
Do the results apply to software people run on other systems?
Not automatically. The reported proofs cover single-threaded static RISC-V binaries linked against musl. They do not establish the same properties for x86 or ARM builds linked against glibc.
Source: ThorstenMeyerAI.com
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
