论文发现基准低分可被人为制造,仅看分数不足以判断模型能力
搞安全评估的都该看看这篇。它做了个 LoRA 拨盘,能让 Qwen 模型精准考出任意低分,误差不到 2 分,但错误集中在同一选项上,破绽露在错误分布里。做模型评测或红队的人直接有用。
一项新研究指出,安全测试中危险能力探测的低分可能并非模型真实能力,而是被人为压低的结果,单看分数无法证明模型安全。
研究团队训练了一个 LoRA "拨盘",把 Qwen 模型拉向一个更小的 Qwen 模型方向,使其在基准上按指定目标失分,同时对比了提示词压制、升高 temperature、加权重噪声等方法。这个 LoRA 拨盘在未见过的目标分数上误差不超过 1.9 分,而提示词方法的偏差高达 42 到 47 分。
但拨盘留下了痕迹:在多项选择题上,它最多 97% 的新错误都落在同一个选项字母上,天然模型池里没有模型接近这一比例。研究者建议,把低分当作低能力之前,应检查模型具体答错哪些题、错误答案如何分布。
This paper finds that a low benchmark score can be manufactured precisely, so a low score alone says little about what a model can do.
Safety tests often read a low score on a dangerous-capability probe as reassurance. But a low score can come from a model being held back, so the number alone proves little.
They trained a LoRA "dial" that pulls a Qwen model toward a smaller Qwen model, then compared its errors with the real small model's. They also tried prompting models to underperform, raising temperature, and adding weight noise.
The dial hit unseen target scores within 1.9 points, while prompting missed by 42 to 47 points. But on multiple choice, up to 97% of its new errors landed on 1 letter, a share no natural model in their pool approached.
Before reading a low score as low capability, check which items the model fails and how its wrong answers are spread.
本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」「中美大模型技术代差与商业化对照」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:Qwen、LoRA、AI安全、基准测试,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「rohanpaul_ai」(@rohanpaul_ai),经 AITOP 多维评分管线结构化提炼。