Skip to content- Sources: primary
- Summary: The study probes successive SFT, DPO and RLVR checkpoints of Olmo-3-7B-Instruct on multi-digit addition, using carry values as known intermediate state. Probes recover carry information equally well after SFT and RLVR, but chains of thought produced by RLVR raise accuracy for every receiving checkpoint more than SFT or DPO ones. DPO shows similar sensitivity to carry statements without comparable accuracy gains.
- Why it matters: It gives evidence that reinforcement learning with verifiable rewards changes how useful written reasoning is, not what the model represents internally.
- Background: RLVR, reinforcement learning with verifiable rewards, trains a model on tasks whose answers can be checked automatically, such as math. Olmo 3 is AI2's open model family released with its intermediate post-training checkpoints, which makes this kind of stage-by-stage comparison possible.