AI 'proved' code correctness: what was actually proven

An article by Sid Trehan (Keller AI) dated November 10, 2026, explains that formal verification of AI code only proves that the code matches a specific mathematical specification under stated assumptions, not that it aligns with business requirements or that it is free from unmodeled effects like timing attacks.

Compiled by Sergey KostenchukPublished 2026-11-11Updated 2026-11-11

2026-11-11 Coding Anthropic

💻 Did AI prove code correctness? Not exactly

Article by Sid Trehan (Keller AI, 11/10/2026): Verification of AI code only confirms compliance with a mathematical specification under stated assumptions, not business requirements or timing attacks.

🌍 Verification is moving from the niche of critical systems into a standard workflow: Anthropic and Theorem Labs generated ~180 million lines of proofs in Rocq using Claude—crash-freedom for 239 entry points in 123 OSS-Fuzz libraries—while incidentally finding hundreds of bugs, including a logical bug that had existed for 10+ years. GitHub Next and Microsoft Research released PDD Kit: an agent writes the code, and Verus checks it against the specification.

👤 A specific property was proven under specific assumptions, not that the code is "bug-free": read the specification and trusted assumptions—PDD Kit includes sign-off cards for SQLite, bcrypt, and JWT.

Source 1: https://kellerai.io/resources/ai-proved-your-code-correct Source 2: https://www.anthropic.com/research/formal-verification

Sources