Skip to content- Sources: primary
- Summary: The authors find that mixed-precision BF16 converges more slowly than FP16 in reinforcement learning with verifiable rewards at the same low learning rate, because small updates to FP32 shadow weights often do not change the BF16 weights used for rollouts and training. Raising the learning rate fixes speed but increases KL divergence from the reference model. Stochastic rounding when casting shadow weights to BF16 matches FP16 convergence across models and datasets while keeping BF16's range and needing no loss scaling.
- Why it matters: Teams running RLVR in BF16 get a cheap change to try before switching to FP16 or raising the learning rate.
- Background: BF16 keeps FP32's exponent range but only 7 mantissa bits, so increments much smaller than a weight's magnitude round away to nothing under round-to-nearest. Stochastic rounding rounds up or down with probability proportional to the distance, so small updates take effect on average. This sits alongside recent work on precision choices in RL post-training, such as FP16 versus BF16 for rollout and training mismatch.