• Sources: paper, discussion
  • Summary: GRPO assigns one trajectory-level advantage uniformly across every policy token, which hides which intermediate decision produced the outcome. ProVer has an agentic judge contrast successful and failed rollouts to nominate the segment responsible for the divergence, then declines to trust that judgment and instead estimates the segment's advantage from the difference in terminal success rates between current-policy continuations sampled before and after it. Reported relative improvements over GRPO across ALFWorld, WebShop, and SearchQA are 9.91% for Qwen3.5-2B and 7.12% for Qwen3.5-4B, submitted 2026-09-28 and the authors' own figures.
  • Why it matters: Using the judge only to choose where to verify avoids evaluating every intermediate state, which is the cost that has kept step-level credit assignment out of agent training runs.