BigSleep vs. Fuzzilli: AI Agent and Fuzzer Find Different V8 Vulnerabilities

🤖 BigSleep vs. Fuzzilli: AI Agent and Fuzzer Find Different V8 Vulnerabilities

Google Project Zero compared the LLM agent BigSleep and the generative fuzzer Fuzzilli on the V8 engine. From April to September 2026, the agent found 45 fixable vulnerabilities compared to 37 for the fuzzer, with a bug overlap of only ~25% (20 out of 81). Of 112 historical bugs, the agent reproduced 62–70% using Fuzzilli.

🌍 First large-scale quantitative measurement: LLM agents compete with mature coverage-guided fuzzing but do not replace it. BigSleep is stronger on complex JIT logic bugs, while Fuzzilli excels in its own class. Vendors should run both methods in parallel.

👤 Useful for those tracking the real capabilities of LLMs: the agent searches rather than cites — no memorization of bugs prior to the model's cutoff was detected. The difference of 45 vs. 37 is statistically inconclusive without a variance assessment.

Source 1: https://projectzero.google/2026/10/fuzzing-v-AI.html Source 2: https://news.ycombinator.com/item?id=50349699