Qwen · Jul 28, 2026
Pass the Baton: Trajectory-Relayed On-Policy Distillation
On-policy distillation (OPD) suffers from prefix failure where student deviations lead to unreliable supervision. Relay-OPD introduces a label-free handoff trigger based on teacher-student continuation asymmetry, constructing relay trajectories where the teacher briefly takes over at critical points. With a Qwen3-4B-Instruct-2507 teacher and Qwen3-0.6B/1.7B-Non-Thinking students, Relay-OPD outperforms standard OPD by +5.73% and achieves best or second-best on eight mathematical reasoning benchmarks.
What happened
Researchers propose Relay On-Policy Distillation (Relay-OPD) to address prefix failure in on-policy distillation. By detecting teacher-student continuation asymmetry as a handoff trigger, the method inserts short teacher legs into student trajectories, concentrating intervention on early critical positions. Experiments with Qwen3 models show consistent gains across eight math reasoning benchmarks, outperforming standard OPD by 5.73%.
Technical significance
Relay-OPD leverages the observation that on failed prefixes, the teacher tends to redirect while the student continues along the original direction. This asymmetry is converted into a label-free trigger for teacher intervention, creating relay trajectories that correct early deviations without full teacher rollouts. The limited relay budget ensures minimal departure from the student policy while improving supervision quality.
Industry impact
This technique could reduce the cost and complexity of distilling reasoning models by making on-policy distillation more robust to student errors, potentially enabling smaller models to achieve higher reasoning performance with less manual curation or expensive teacher calls.
What to watch
Next signals include application to larger models, integration with other distillation methods, and testing on broader reasoning domains beyond math. The label-free trigger mechanism may also inspire new approaches to student-teacher alignment in reinforcement learning.
Decision value
Improves the efficiency and effectiveness of model distillation, which is critical for deploying capable reasoning models in resource-constrained environments. Could lower inference costs and enable on-device reasoning capabilities.