用 Jev 单条是非题检测对齐失败：中位 AUROC 0.886
一篇省钱的对齐检测论文：问 Jev 一个是非题就能给回复打分，成本只有 LLM 裁判的约 1/60，还顺带找出三个基准的标注错误。
论文提出用 TypeSafe AI 的校准决策模型 Jev 提问一条通用是非题，以输出概率作为评分来识别模型对齐失败，无需额外训练即可达到中位 AUROC 0.886。成本方面，在 19 个基准上 Jev 单次通过花费 $0.30，而这些基准常用的 LLM 裁判要 $18.96。研究者构建了 RLCDAlignBench，整合 44 个现有基准、覆盖谄媚、越狱、欺骗、提示注入、奖励劫持等十类失败。在 StrongREJECT 上，Jev 与人工标注的一致性达到 GPT-4o-mini 评分器水平，排序能力更强（AUROC 0.971 对 0.929）。研究还发现阈值不能跨基准迁移，在 10 条标注数据上拟合阈值可将中位 F1 从 0.706 提升到 0.793。
Cool paper on catching alignment failures with Jev. The ideas is to ask Jev one generic yes/no question about a model's response, and use its probability as a score. With no extra training, that score separates failures from good responses well, with a median AUROC of 0.886. It is also cheap. On 19 benchmarks, a Jev pass cost $0.30, while the LLM judges those benchmarks use cost $18.96. Researchers built RLCDAlignBench from 44 existing benchmarks across ten failure types, including sycophancy, jailbreaks, deception, prompt injection and reward hacking. Jev, TypeSafe AI's calibrated decision model, answers many typed questions about one input in a single call, each with a probability. On StrongREJECT, it agrees with human labels as well as the GPT-4o-mini scorer does, and ranks responses better (AUROC 0.971 vs 0.929). Where Jev confidently disagreed with benchmark labels, it found label errors in three benchmarks. They find that the question wording matters little. The thresholds do not transfer between benchmarks, and fitting one on 10 labelled items raises median F1 from 0.706 to 0.793. Paper: arxiv.org/abs/2609.29429 Chat with Paper: academy.dair.ai/papers/just-as… 💬 3 🔄 1 ❤️ 7 👀 1067 📊 4 ⚡
- OpenRouter09-25 22:21原文
