研究显示：简单特征胜过多数 AI Control 监控模型
Simple Features Beat Most AI Control Monitors
如果你在做 AI Control 或监控器评估，这篇得看：它指出了标准数据集的两个漏洞，还重建了数据集和检查工具一并开源。
一项针对 AI Control 监控器的评估研究发现，在标准评估数据集上，不运行代码的简单特征就能 outranking 24 个用作监控器的开源模型中的 20 个。原因有二：数据集中诚实代码由人类编写、后门代码由较老的语言模型编写，代码风格本身泄露了标签；同时 LLM 能植入后门的题目也带有可识别特征。用 Claude Sonnet 5 的后门替换一类代码后，人类代码对 Sonnet 5 后门的区分度达到 0.95，高于原始配对的 0.79，说明这些特征主要是在区分作者而非检测破坏。研究还发现标准测试集中七分之一的口径后门未通过有效性检查，Sonnet 5 监控器对老模型后门的捕获率从表面上的 77% 降到验证后的对比 78% vs 61%。
Simple Features Beat Most AI Control Monitors
AI control relies on monitors to catch a capable model that sabotages its work. These monitors are evaluated on their ability to differentiate honest from backdoored code. We show that on a standard evaluation dataset, simple features beat most monitors without ever running the code, outranking 20 of 24 open-weight models prompted as monitors. The dataset gives its labels away in two ways. People wrote the honest solutions and older language models the backdoors, so the style of the code reveals the class. Additionally, problems that LLMs can backdoor have different characteristics, so basic facts about the problem reveal it too. Swapping one class at a time for code by Claude Sonnet 5 shows that these features reward a contrast between authors, since human code against Sonnet 5's backdoors is easier to separate than the original pairs, 0.95 against 0.79. One in seven backdoors in the standard test set fails our validity checks, which distorts comparisons between attackers. A Sonnet 5 monitor appears to catch 77% of the older models' backdoors but 37% of Sonnet 5's, and 78% against 61% once both are verified. We do not show that published monitors rely on these shortcuts, but our findings suggest that a high score on such a dataset is not by itself evidence of sabotage detection. We release the rebuilt dataset and checks.
