• Sources: primary
  • Summary: The paper measures quantization, weight staleness and injected noise on one scale across three student-teacher pairs and finds on-policy distillation tolerates substantial training-inference mismatch up to an empirical budget. It proposes an admission check that screens rollout configurations against that budget without a full training run. Model-specific FP8 recipes hold at least 1.93 times as many KV-cache tokens as BF16, and FP8 with asynchronous rollouts reaches the same quality 1.62 times sooner than synchronous BF16.
  • Why it matters: Teams distilling into smaller open-weight models can use cheaper quantized and asynchronous rollout engines with less risk than in RL, after checking sensitive layers.
  • Background: On-policy distillation has the student generate its own responses and trains it to match a fixed teacher's token distribution on them, which avoids the train-test gap of distilling on teacher text. Rollout engines such as vLLM often run in lower precision or on stale weights for speed, so the policy that generated data differs from the one being trained. In RL this mismatch is a known source of instability.