SteerDistill：用教师闭环引导量化 LLM Agent，W4A16 下逼近 BF16
SteerDistill: Quantizing LLM Agents via Closed-Loop Teacher Steering
如果你在部署低比特量化的 Agent，这篇很值得看。它让 W4A16 量化模型的表现只差 BF16 一分以内，训练成本还比 QeRL 少 67% 的 GPU 时数。做端侧 Agent 或成本敏感推理的工程师可以直接参考。
论文提出 SteerDistill，一种量化感知的蒸馏方法，解决低比特量化导致 LLM Agent 任务成功率下降的问题。作者分析失败轨迹后发现，量化后的 Agent 会遗漏任务要求或环境条件，导致环境状态转移偏离全精度参考轨迹。
方法上，全精度教师引导学生修正失败响应，并利用学生执行反馈迭代调整引导，形成闭环教师引导；训练时让学生在修正回合匹配引导后学生的预测、其余回合匹配全精度教师，实现推理时无需引导。
在 5 个 agentic 基准和 3 个模型上，SteerDistill 在 W4A16 下平均得分超过最强量化基线 1.67–2.18 分，与 BF16 差距小于 1 分；在 τ²-Bench 上成功率比 QeRL 高 1.84 个百分点，训练 GPU 时数减少 67%，成功轨迹也更短。
SteerDistill: Quantizing LLM Agents via Closed-Loop Teacher Steering
Low-bit quantization reduces the inference cost of LLM agents but can compromise task success. Examining failed rollouts, we find that quantized agents overlook task requirements or environmental conditions, producing environment transitions that diverge from successful full-precision references. Correcting these behavioral divergences remains difficult: offline data may miss the student's failure contexts, while its own rollouts may provide few correct responses in those contexts. We introduce SteerDistill, a quantization-aware distillation method in which a full-precision teacher guides the student to revise failed responses and iteratively refines its guidance using feedback from student execution. This closed-loop teacher steering produces locally verified behavioral corrections. We internalize these corrections by training the student without guidance to match the steered student's predictions at corrected turns and the full-precision teacher's at other turns, enabling unassisted inference. Across five agentic benchmarks and three models, SteerDistill exceeds the strongest quantized baseline by 1.67--2.18 points in average score under W4A16 and stays within one point of BF16. On $τ^2$-Bench, it outperforms QeRL by 1.84 percentage points in success rate using 67% fewer training GPU hours and produces shorter successful trajectories than the quantized baselines.
本条情报归属 AITOP「论文」核心栏目，重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：SteerDistill、QeRL、τ²-Bench、量化，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「arXiv cs.LG」（@Genming Cui, Xiangyu Li, Zixu Hao, Xin Ding, Yunxin Liu, Ting Cao），经 AITOP 多维评分管线结构化提炼。