AutoRL:用智能体监督强化学习训练,Qwen3 AIME 成绩提升 20 个百分点
把 RL 训练调参交给智能体团队:Perceptor 看曲线、Proposer 出方案、Orchestrator 执行热更新。Qwen3 数学基准涨 20 个百分点,连故意写坏的配方都能自动修。做训练管线或想省调参人力的人值得细看。
一篇新论文提出 AutoRL 框架,让 AI 智能体接管强化学习训练的实时监督,替代固定人工流程。在 Qwen3 上,AIME 2024 准确率从 46.67% 提升到 66.67%,AIME 2025 与 HMMT 2025 各提升 11.67 分。
AutoRL 的架构分三层:Perceptor 智能体持续观察训练曲线、rollout 与评估结果并标记异常;三个 Proposer 智能体诊断问题并提出修改方案;Orchestrator 审批后通过热更新或从 checkpoint 重启来应用改动。在 agentic coding 场景中,配合 LongCat-Flash-Lite,pass@1 从 52.29% 升至 65.13%,且是在长时间无人值守、包含中断的运行中完成的。作者还故意给出一个有缺陷的训练配方,AutoRL 成功找到并修复了问题。
论文指出,智能体已能胜任常规的 RL 监督工作,但每次改动仍需大量算力来验证,这是当前的主要瓶颈。
Self-improving AI is one of the most discussed ideas right now. This is one of the most exciting papers I've read recently on the topic. (bookmark it) It proposes AutoRL, which lifts Qwen3 on AIME 2024 from 46.67% to 66.67% by letting agents supervise a live RL run instead of following a fixed recipe. AutoRL works like an RL researcher on call. Perceptor agents watch training curves, rollouts, and evaluations and flag problems. Three Proposer agents diagnose the issue and suggest changes. An Orchestrator approves them and applies them with a hot update or a restart from a checkpoint. On math, AIME 2025 and HMMT 2025 each improve by 11.67 points. On agentic coding with LongCat-Flash-Lite, pass @1 rises from 52.29% to 65.13% over a long unattended run with interruptions. Given a deliberately broken recipe, AutoRL found and fixed the problems. The authors say agents can already handle routine RL supervision. Validating each change still takes a lot of compute. arxiv.org/abs/2611.17790 Chat with Paper: academy.dair.ai/papers/autorl-…📊 15 ⚡
本条情报归属 AITOP「论文」核心栏目,重点聚焦中国 AI 产业与国产大模型生态。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:AutoRL、Qwen3、强化学习、智能体,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「elvis」(@omarsar0),经 AITOP 多维评分管线结构化提炼。