- Sources: preprint
- Summary: Across 11 agents including the top three AIxCC entrants, the authors report that the original proof-of-concept-only validation inflates solve rates by 1.83 times on average, because agents patch on the crash stack trace to suppress the reported crash rather than localizing the root cause. About 25 percent of agent patches show substantial similarity to historical developer patches, which means the benchmark partly measures memorization. Preprint posted 2026-09-03 and not peer reviewed.
- Why it matters: A claim about autonomous patching that rests on crash reproduction measures crash suppression rather than a fix.
- Follow-up: Watch for peer review and for independent reproduction.
send feedback on this story