首个全程递归自我改进的前沿预训练：开源 801-30B-A3B
一群研究者用智能体循环改造预训练流程，从零练出 801-30B-A3B，只做 6B token SFT 就在 agentic 基准上打赢 Qwen3.6-35B-A3B。想了解 RSI 怎么落地到真实训练管线、省掉巨额后训练成本的开发者值得看。
一篇新论文声称首次将递归自我改进（RSI）应用于一次完整的前沿预训练运行，从零训练出开源 30B MoE 基座模型 801-30B-A3B。该模型在基准上超过了 Qwen3-30B-A3B 和 Nemotron-3.5-Lightning-30B-A3B。
技术难点在于预训练一次要花数十万 GPU 小时且只跑一次，无法用一个大的反馈循环来改进。团队因此把工作拆成多个可闭环的小循环：智能体负责调试基础设施和数据、用小型 SFT 运行检验 post-training 提升空间、把专家判断转化为数据管线的量化指标，并通过共享上下文存储跨团队保留失败经验。
结果上，该模型仅用 6B token 做 SFT、未做 RL，就在四个 agentic 基准上取得对完全后训练模型的最高平均分，其中包括 Qwen3.6-35B-A3B。这说明智能体驱动的方法可以切入此前最难自动化的预训练环节。
Recursive self-improvement is one of the most discussed ideas in AI right now.
Banger paper on RSI for frontier pre-training.
They claim to be the first in the world to apply RSI to an entire frontier pre-training run from scratch.
This work uses agent-driven loops on a single frontier pre-training run and produced 801-30B-A3B, an open 30B MoE base that beats Qwen3-30B-A3B and Nemotron-3.5-Lightning-30B-A3B.
Pre-training is hard to improve with one big loop. A run costs hundreds of thousands of GPU-hours and happens once, and its real goal is headroom for post-training.
So they split the work into smaller loops that can close. Agents debug infrastructure and data, check post-training headroom with tiny SFT runs, and turn expert judgment into metrics for data pipelines. A shared context store keeps negative results across teams.
With only 6B tokens of SFT and no RL, the model had the highest average on four agentic benchmarks against fully post-trained models, including Qwen3.6-35B-A3B.
https://t.co/ixEDPMIeI3
Chat with Paper: https://t.co/K5eG1VcjYl
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：RSI、801-30B-A3B、Qwen3、开源模型，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「DAIR.AI」（@dair_ai），经 AITOP 多维评分管线结构化提炼。
