NYU与Amazon研究技能蒸馏优化方法
NYU和Amazon发现，选对6个技能比盲目使用71个技能更有效，Qwen3-8B准确率提升2%。
NYU和Amazon联合发布新论文，提出SGUID方法筛选技能文件。研究显示，仅保留能持续提供训练信号的少量技能，效果可媲美多达11倍大小的技能库。在3个Qwen模型测试中，仅25%的技能提供有用信号。使用6个精选技能后，3个模型在数学竞赛测试中表现超越完整技能库。第二轮添加3个新技能后，Qwen3-8B准确率从64.3%提升至66.3%。
Good paper for selecting your skill files. Before distilling a skill bank, log which skills give a steady training signal and drop the rest.
New NYU and Amazon paper finds that distilling a few skills that keep producing a useful training signal matches or beats distilling a skill bank up to 11× larger.
Skills are short written tips, like a rule for counting cases, that a model absorbs by learning from a copy of itself that reads them. Picked by topic match, under 25% of them gave any useful signal across 3 Qwen models.
SGUID keeps only skills that help early in training and still help late. With 6 such skills, 3 of 4 models matched or beat the full bank of 30 to 71 skills on math contest tests. A 2nd round with 3 new skills lifted Qwen3-8B from 64.3% to 66.3%.