Endor Labs benchmark reports mid-table security-fix results for Claude Fable 5

Category: Pulse
Status: discussion
Sources: Endor Labs post, HN discussion
Requested: reader inbox (#11)
Summary: Endor Labs ran Claude Fable 5 on its Agent Security League benchmark of 200 real-world vulnerability-fixing tasks and reports 59.8% functional pass and 19.0% security pass, mid-table on its leaderboard, with 15 timeouts past 40 minutes and memorization-based answers detected on 38 of 200 instances. The post also notes Fable 5 fixed four vulnerabilities no prior model-agent combination had solved. The authors state their benchmark measures safe code authoring, a different axis than Anthropic's exploit-focused headline evaluations. The HN thread (305 points) debates benchmark validity against launch claims.
Why it matters: Independent task-level results with published method give a counterweight to launch benchmarks when choosing models for security-sensitive code work.