Senior SWE-Bench scores coding agents on senior-level tasks

Category: Agentic coding
Status: discussion
Sources: Senior SWE-Bench, dataset, HN discussion
Summary: Researchers at Princeton and the University of Wisconsin-Madison with Snorkel AI published Senior SWE-Bench, an open benchmark (dataset snorkel-ai/senior-swe-bench-v2026.06) that evaluates coding agents on 50 tasks across more than 15 repositories in Python, TypeScript, Go, Rust, Elixir, and other stacks. Tasks use natural-language instructions rather than fully specified requirements, bug and performance tasks are drawn from pull requests that required runtime investigation from logs and behavioral reports, and a "taste" score combines correctness tests with code-quality metrics derived from each codebase's own conventions. Reported solve rates are Claude Opus 4.8 at 24.0 percent, Claude Sonnet 5 at 19.4 percent, and GPT-5.5 at 16.0 percent, and the authors state frontier models fail senior-level correctness on more than 75 percent of tasks.
Why it matters: It measures agents on under-specified, investigation-heavy work closer to real senior engineering, and the low solve rates quantify how far current agents remain from autonomous senior-level output.
Follow-up: Watch for independent reproduction of the solve rates and coverage of additional models. Cursor also published its own CursorBench 3.1 eval the same day (vendor-run).