用 Jev 单条是非题检测对齐失败：中位 AUROC 0.886

886。成本方面，在 19 个基准上 Jev 单次通过花费 $0.

30，而这些基准常用的 LLM 裁判要 $18.96。

研究者构建了 RLCDAlignBench，整合 44 个现有基准、覆盖谄媚、越狱、欺骗、提示注入、奖励劫持等十类失败。在 StrongREJECT 上，Jev 与人工标注的一致性达到 GPT-4o-mini 评分器水平，排序能力更强（AUROC 0.

971 对 0.929）。

研究还发现阈值不能跨基准迁移，在 10 条标注数据上拟合阈值可将中位 F1 从 0.706 提升到 0.

793。

Cool paper on catching alignment failures with Jev. The ideas is to ask Jev one generic yes/no question about a model's response, and use its probability as a score. With no extra training, that score separates failures from good responses well, with a median AUROC of 0.886. It is also cheap. On 19 benchmarks, a Jev pass cost $0.30, while the LLM judges those benchmarks use cost $18.96. Researchers built RLCDAlignBench from 44 existing benchmarks across ten failure types, including sycophancy, jailbreaks, deception, prompt injection and reward hacking. Jev, TypeSafe AI's calibrated decision model, answers many typed questions about one input in a single call, each with a probability. On StrongREJECT, it agrees with human labels as well as the GPT-4o-mini scorer does, and ranks responses better (AUROC 0.971 vs 0.929). Where Jev confidently disagreed with benchmark labels, it found label errors in three benchmarks. They find that the question wording matters little. The thresholds do not transfer between benchmarks, and fitting one on 10 labelled items raises median F1 from 0.706 to 0.793. Paper: arxiv.org/abs/2609.29429 Chat with Paper: academy.dair.ai/papers/just-as…📊 4 ⚡

本条情报归属 AITOP「论文」核心栏目，重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」「各国 AI 监管法案、合规治理与主权算力政策」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：Jev、TypeSafe AI、RLCDAlignBench、StrongREJECT，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明：本文原始线索采编自「elvis」（@omarsar0），经 AITOP 多维评分管线结构化提炼。截至目前，全网已有 3 家独立信源对该事件进行交叉印证与跟进报道（含 OpenRouter、a16z 等），属于近期高关注度行业信号。