Stanford 论文:Qwen3.5-27B 无梯度自我改写权重,准确率 57.2% 升至 63.8%
27B 模型不改训练流程、靠对比自身隐藏状态就涨了 6.6 个百分点,这事以前只有重训才能做到。有趣的是规模很关键,小模型会把自己改坏。自己部署开源大模型的团队可以直接照论文方法试试。
Stanford 与多家顶级实验室联合发表论文,发现 27B 开源模型可以在无训练、无梯度的情况下通过推理自行修改权重来提升能力。研究团队给 Qwen3.5-27B 提供自身的权重、带评分的练习答案和一个 Python 会话,并屏蔽梯度。模型对比自己在完成答案与耗尽 token 答案上的隐藏状态,然后削弱区分两者的方向,完成自我修正。
在一组预先固定的 490 道未见题目测试中,这次自我编辑将准确率从 57.2% 提升到 63.8%。但该方法对规模敏感:2B 模型没有做出任何有效编辑,4B 和 9B 模型则改坏了自己。
这项工作挑战了'改进已部署模型必须重新训练'的常规做法——传统上提示词和额外推理计算的收益在任务结束后即消失,而权重自编辑的改进是持久的。对运行大型开源模型的团队,论文建议让模型针对自身失败案例提出权重编辑,但只保留能通过留出测试的编辑。
New Stanford+other top labs paper finds that a 27B open model can improve itself by editing its own weights through reasoning, with no training or gradients.
Improving a deployed model today means another training run on large clusters. Prompts and extra inference compute help, but the gain disappears once the task ends.
They gave Qwen3.5-27B its own weights, its graded practice answers, and a Python session, with gradients blocked. It compared its hidden states on answers that finished against answers that ran out of tokens, then weakened the direction separating them.
Its own edit raised accuracy from 57.2% to 63.8% on 490 unseen problems, in a test fixed in advance. The 2B model made no edit, and the 4B and 9B models broke themselves.
If you run a large open model, let it propose weight edits for its own failures, but keep only edits that pass held-out tests.
本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:Qwen3.5、Stanford、权重编辑、开源模型,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「rohanpaul_ai」(@rohanpaul_ai),经 AITOP 多维评分管线结构化提炼。截至目前,全网已有 2 家独立信源对该事件进行交叉印证与跟进报道(含 Genspark 等),属于近期高关注度行业信号。
- Genspark11-19 04:12原文