NVIDIA 公开智能体编程中间训练配方，SWE-bench Multilingual 提升 7 分
NVIDIA 开源了一个中间训练配方，给 Nemotron 加一段 500B token 的软件工程数据训练，SWE-bench 多语言直接从 55.8 涨到 62.8，自己训练编程智能体的可以照抄。
NVIDIA 发布一份开放的中间训练（midtraining）配方，用于提升智能体编程能力。该阶段位于预训练与后训练之间，在 256K 上下文上用 500B token 的自然软件工程数据训练，包括打包仓库、pull request、commit 历史和构建日志，外加少量 agent traces。在 SFT 保持一致的前提下，Nemotron 3.5 Lightning 的 SWE-bench Multilingual 从 55.8 提升到 62.8，SWE-bench Verified 从 60.7 提升到 64.5，SWE-bench Pro 从 32.0 提升到 35.5。更大的 Nemotron 3.5 Super 上 DeepSWE 从 11.7 提升到 15.7。由于基座模型的代理指标无法预测后训练结果，团队用短固定 SFT 运行来评分每个数据选择。
Great paper from NVIDIA sharing an open midtraining recipe for agentic coding.
NVIDIA kept SFT identical and added one midtraining stage. SWE-bench Multilingual went from 55.8 to 62.8 on Nemotron 3.5 Lightning.
The stage sits between pretraining and post-training. It trains at 256K context on 500B tokens of natural software-engineering data, such as packed repositories, pull requests, commit histories and build logs, plus a small share of agent traces.
On the same model, SWE-bench Verified rose from 60.7 to 64.5 and SWE-bench Pro from 32.0 to 35.5. On the larger Nemotron 3.5 Super, DeepSWE rose from 11.7 to 15.7.
They also explain how they evaluated the recipe. Base-model proxy metrics did not predict post-training results, so every data choice was scored with short fixed SFT runs.
If you train coding agents, the recipe and its ablations are published.
https://t.co/SJ7JHdd1HY
Chat with Paper: https://t.co/eCK8iUNIi0
