• Sources: primary
  • Summary: The authors find FP4 recovery recipes that work for dense models give no gain over naive FP4 on mixture-of-experts RL post-training. Asymmetric corrections lower the correlation of training and inference probabilities and router selection match rates. MoeResQ enforces symmetric correction and largely closes the gap to full precision.
  • Why it matters: Teams running low-precision RL on MoE models should check router agreement between trainer and rollout engine, not just token probabilities.