研究重新审视“RL 泛化、SFT 记忆”：SFT 数据质量与规模是关键

在数据规模上，当 SFT 与 RL 匹配正确 rollout 数量时，SFT 泛化能力大幅提升；匹配总 rollout 预算时，SFT 能达到与 RL 相当的泛化水平；两者结合后 SFT 甚至超过 RL。作者还用 LLM 标注分析训练 rollout 中的解题方法，发现更大数据集覆盖更多尾部方法，这些尾部方法提供了可泛化的推理信号。

论文最后通过分析浅层 Transformer 在 RL 与 SFT 下的训练动态，为上述实证结论提供理论支撑。

Rethinking "RL Generalizes, SFT Memorizes": The Role of SFT Data

Large language models trained with Reinforcement Learning (RL) with verifiable rewards exhibit strong reasoning ability and broad generalization, whereas models trained with Supervised Fine-Tuning (SFT) are often viewed as more prone to memorization and limited transfer. This paper rethinks this distinction through the lens of SFT training data on widely used Qwen2.5 and Qwen3 series models. First, we study the role of data source and show that it is critical: a carefully mixed SFT dataset substantially outperforms data generated solely by a larger model. Second, we study the role of data scale and show that matching the number of correct rollouts between SFT and RL greatly improves SFT generalization, while matching the total rollout budget enables SFT to generalize as well as RL. Combining these two factors further enables SFT to generalize even better than RL. Third, by using LLM annotations to characterize the solution methods in training rollouts, we show that larger datasets cover more tail methods and that these tail methods provide generalizable reasoning signals. Finally, we support these empirical findings theoretically by analyzing the training dynamics of shallow transformers under both RL and SFT.

本条情报归属 AITOP「论文」核心栏目，重点聚焦中国 AI 产业与国产大模型生态。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：SFT、RL、Qwen2.5、Qwen3，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明：本文原始线索采编自「arXiv cs.LG」（@Yunlong Hou, Fengzhuo Zhang, Yuan Cheng, Jiachun Pan, Xingyao Li, Zhuoran Yang），经 AITOP 多维评分管线结构化提炼。截至目前，全网已有 2 家独立信源对该事件进行交叉印证与跟进报道（含 elvis 等），属于近期高关注度行业信号。