NVIDIA Nemotron 3.5 Super 用专家构建任务做 RL 后训练的实验报告
What RL on Expert-Built Tasks Teaches Nemotron 3.5 Super
NVIDIA 把五个领域分开各训一个 LoRA 适配器,十二个基准最高涨 13.8 分,SWE-bench Pro V2 涨到 92.1%。做法是纯 RL 不带监督阶段,还附了失败模式逐条分析,做后训练的工程师能直接抄作业。
NVIDIA 在 arXiv 发布报告,对 Nemotron 3.5 Super(120B 总参数、12B 激活参数)进行强化学习后训练。实验使用五套专家构建的任务数据集,逐个训练:真实软件工程、复杂约束、长程工具调用、专业图表理解和企业文档推理。每次训练基于 Tinker 平台,用 CISPO 算法训练 rank-32 LoRA 适配器,全程无监督微调阶段,并与基座模型在相同任务、同一评测框架下对比。
在十二个基准上(八个由第三方构建、四个由 Surge AI 构建),提升幅度从 IFBench 的 +4.5 分到 GDP_pdf 的 +13.8 分不等,其中 GDP_pdf 从 10.4% 升至 24.2%。其他关键结果包括:SWE-bench Pro V2 从 86.9% 升至 92.1%,Terminal-Bench 2.1 从 63.9% 升至 70.3%,MultiChallenge 从 58.3% 升至 66.6%,AutomationBench 从 14.8% 升至 27.5%,Chartography 从 39.4% 升至 50.6%。值得注意的是,文档适配器虽未使用 API 工具训练,但在 AutomationBench 上仍获得 +8.0 分。
报告还做了逐条对照分析:基座模型的失败集中在少数几种模式,而各适配器补上了对应短板——工具调用适配器先找数据和规则再行动,编程适配器在 SWE-bench Pro V2 上自行编写测试或草稿文件,约束适配器会把前几轮对话内容带入回答,图表适配器用代码测量图形标记,文档适配器则真正读取 AutomationBench 的电子表格。
What RL on Expert-Built Tasks Teaches Nemotron 3.5 Super
We post-train NVIDIA's Nemotron 3.5 Super (120B total, 12B active parameters) with reinforcement learning (RL) on five expert-built task datasets, one at a time: real-world software engineering, complex constraints, long-horizon tool use, professional chart understanding and enterprise document reasoning. Each run trains a fresh rank-32 LoRA adapter with CISPO on Tinker, with no supervised stage, and each adapter is compared with the base model on the same tasks, harness and settings. On twelve headline benchmarks, eight built by other groups and four by Surge AI, the gains range from +4.5 points on IFBench to +13.8 on GDP_pdf, which rises from 10.4% to 24.2%; SWE-bench Pro V2 rises from 86.9% to 92.1%, Terminal-Bench 2.1 from 63.9% to 70.3%, MultiChallenge from 58.3% to 66.6%, AutomationBench from 14.8% to 27.5% and Chartography from 39.4% to 50.6%. The document adapter, trained without API tools, also gains +8.0 points on AutomationBench. In attempts read side by side with the base model's on tasks each adapter newly solves, the base model's failures recur in a few modes, and each adapter does more often what the base model missed: the tool-use adapter finds the data and the rules before acting, the coding adapter writes its own test or scratch files on SWE-bench Pro V2, the constraints adapter carries earlier turns into its answers, the chart adapter measures the marks in code, and the document adapter reads AutomationBench's spreadsheets.
本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:Nemotron、LoRA、强化学习、SWE-bench,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Sushant Mehta, Edwin Chen),经 AITOP 多维评分管线结构化提炼。