ActionOPD：动作级在线蒸馏方法，让小模型学会智能体决策
ActionOPD: Distilling Decisions, Not Tokens
把强智能体蒸馏进小模型的新方法，不要求师生同族模型，Qwen3-4B 实测涨 3.3-6.9 个点，做 agent 压缩的可以看看。
论文提出 ActionOPD，一种动作级在线蒸馏方法，用语义动作而非 token 作为监督单元。它在学生模型遇到的状态上同时从学生和黑盒教师采样动作，按语义一致性奖励学生动作，无需教师 token 概率或相同分词器。用 Qwen3-4B 和 Qwen3-8B 配合两个教师训练后，在四个分布外基准上平均提升 3.3-6.9 个百分点。在 τ²-bench 上，ActionOPD 与离线蒸馏和 token 级 OPD 表现相当。论文还探索了 Self-ActionOPD，用同模型教师加特权上下文进行训练。
ActionOPD: Distilling Decisions, Not Tokens
Distilling capable language-model agents into smaller models requires aligning the student's behavior with the teacher's. This raises a fundamental question: at which states and at what granularity should this alignment be performed? Offline imitation provides dense but token-specific supervision and can suffer from state-distribution mismatch. Token-level on-policy distillation (OPD) addresses this mismatch but typically requires teacher token probabilities and compatible tokenization. We argue that semantic actions provide a natural unit of supervision for agent distillation. We introduce ActionOPD, an action-level on-policy distillation method motivated by maximizing expected teacher-action likelihood at student-encountered states. At each such state, ActionOPD samples actions from both the student and a black-box teacher, rewarding student actions according to their semantic agreement with the sampled teacher actions. Through policy optimization, ActionOPD aligns the student's action-level behavior with the teacher's decisions without requiring reproduction of teacher-written reasoning or access to teacher token probabilities. Training Qwen3-4B and Qwen3-8B with two teachers on diverse, curated tool-use tasks yields average gains of 3.3-6.9 percentage points across four out-of-distribution benchmarks. ActionOPD scales favorably with student size and teacher capability, without requiring the teacher and student to share a model family or tokenizer. On $τ^2$-bench, ActionOPD achieves competitive performance against offline distillation and token-level OPD. We further explore Self-ActionOPD with a same-model teacher and privileged context. These results establish ActionOPD as a promising approach to agent distillation.
