Ambient SFT:用高温分布匹配降低微调过拟合
Ambient LLMs: Data-Efficient Supervised Fine-tuning via Temperature Matching
做微调的人可以看看这篇:它把预训练模型自己当噪声老师,只要求高温分布粗略对齐,数据少也能压住过拟合。比标准 SFT 平均相对提升最高 21%,介于 SFT 和蒸馏之间可调,适合数据不够多的后训练场景。
arXiv 论文提出 Ambient SFT,一种利用不完美预测扩充微调数据的方法,旨在解决 SFT 数据量小于预训练数据导致的过拟合和长尾分布灾难性遗忘问题。核心机制是用冻结的预训练模型作为噪声教师,要求微调后模型在高温条件下的条件分布与预训练模型的高温分布匹配,温度超参数控制对预训练模型的信任程度:T 趋于无穷时退化为标准 SFT,T=1 时等价于知识蒸馏。
实验覆盖领域适配、指令微调和推理三类任务,相对标准 SFT 平均相对提升最高达 21%。该方法不需要额外的大规模数据收集,直接复用预训练模型自身作为教师信号。
对于做后训练的团队,这提供了一条在数据稀缺场景下抑制遗忘的路线,介于普通 SFT 和蒸馏之间的连续可调设计也便于按数据质量折中取舍。
Ambient LLMs: Data-Efficient Supervised Fine-tuning via Temperature Matching
Collecting large-scale, diverse, and high-quality post-training datasets for autoregressive language models is challenging. Consequently, Supervised Fine-Tuning (SFT) is typically applied to datasets significantly smaller than those used for pre-training, which leads to overfitting and the catastrophic forgetting of the tails of the data distribution. To prevent this collapse, we introduce Ambient SFT, a principled framework for leveraging imperfect predictions to augment the fine-tuning set. In our method, a "frozen" version of the pre-trained model serves as a noisy teacher that the fine-tuned model must match at a coarse level. In particular, we require that the high-temperature versions of the conditional distributions produced by the post-training model should match the high-temperature versions of the pre-training ones. The temperature hyperparameter controls the level of trust in the pre-trained model, with the limit $T\to \infty$ reverting back to regular SFT and $T=1$ to knowledge distillation. We demonstrate that our approach outperforms standard SFT across domain adaptation, instruction-tuning, and reasoning, leading for up to $21\%$ average relative improvement to standard SFT.
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:SFT、微调、知识蒸馏、ArXiv,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Giannis Daras, Marios Papachristou, Adam Klivans, Antonio Torralba, Constantinos Daskalakis),经 AITOP 多维评分管线结构化提炼。