• Sources: primary
  • Summary: Andrew Gao and William Gao show that round-to-nearest conversion from fp32 master weights to bf16 rollout weights drops most per-step updates at common learning rates. Their Fair Cast fix exports with stochastic rounding or error feedback and computes loss on the served copy, costing 22 to 38 ms per 1.7B export. Across 25 matched seed pairs on Qwen3-0.6B to 4B it was ahead on 18, tied on 6 and behind on 1, with a mean gain of 6.1 points.
  • Why it matters: Teams running RL post-training with fp32 masters and bf16 inference engines may be generating rollouts from a stale policy without noticing.