Skip to content- Sources: primary
- Summary: The authors show through a teacher-intervention study that the preferred self-teacher varies token by token within one response, identified by majority voting and KL-based selection. MTA-OPSD builds a pool of candidate teachers with different reference exposure and selects one per token during training. On Qwen3 models for math reasoning it beats prior OPSD methods with an average gain of up to 5.9 over the base model, and code is released.
- Why it matters: It adds a token-level knob to self-distillation recipes that post-training teams can test against fixed-teacher OPSD.