• Sources: primary
  • Summary: The paper shows that at the KL-regularized optimum RLVR only reweights the accepted set, so the hack rate is inherited from the base model, and drift from that comes from parameter sharing across prompts. With sharing, a wrong answer accepted by most prompts is amplified and one accepted by few is pushed below the base rate, under Adam, SGD and sampled GRPO. Two verifier patches removing equally many errors differed by up to 38 points of true accuracy.
  • Why it matters: When fixing a verifier, prioritize the false positives the model repeats across many prompts rather than counting errors removed.