• Sources: primary
  • Summary: SnowPR has 65 tasks from merged PRs, 62 of them in Snowflake's Bazel monorepo with 17 million lines of nontest Java and C++. A trial passes only if every hidden test from the original PR passes. With opus-5, four harnesses passed 84.1% to 85.1% at $6.34 to $8.19 per trial. Java and C++ tasks passed 55% of trials against 80% for Java alone.
  • Why it matters: Harness choice moved cost more than pass rate here, and cross-language changes remain where agents fail most often.