研究：短文本 AI 检测器误报率随长度骤变，提出分长度校准方法
How Short Is Too Short? Length-Conditional Calibration and Abstention for Detecting AI-Generated Text
做内容审核或学术诚信检查的会想看这篇：它用公式证明检测器在 10 token 短文本上误报率会从 1% 飙到 29%，Binoculars 真阳性率最高也只有 59%，还给了分长度校准和自动弃权的具体做法，可以直接照着实现。
一篇 arXiv 论文（arXiv:2611.10901）研究 AI 生成文本检测器在短文本上的失效问题。作者给出闭式公式，描述长文本上设定的阈值在文本变短时误报率如何变化，以及检测器达到目标真阳性率所需的最少 token 数。论文用 conformal p 值按长度分桶校准阈值，并加入分桶弃权规则：检测器在该长度下不具信息量时直接拒绝判定。
在三个零样本检测器（log-likelihood、log-rank、Binoculars）、三个语料库和两个开源权重生成模型上的实验显示，分桶校准在 10 到 200 token 的每个检测器-长度单元格中把平均误报率稳定在 5.0%，基于 300 次随机文档级重划分取均值。相比之下，单一在 200 token 设定的阈值在 10 token 处让 Binoculars 误报率达 29%，log-likelihood 约 1%，与闭式公式预测一致。
论文还揭示检测能力的天花板：没有任何检测器在短文本上达到 80% 真阳性率，最好的 Binoculars 在 200 token 处也只有 59%。因此按 80% 底线时规则在每个单元格都弃权；底线降到 0.3 和 0.5 时，才只放出 200 token 的分桶结果。
How Short Is Too Short? Length-Conditional Calibration and Abstention for Detecting AI-Generated Text
Detectors of AI-generated text are usually tuned and tested on passages of several hundred tokens, yet many real cases are short. We study score-based detection as a function of passage length. We give a closed form for how the false-positive rate of a threshold set on long text changes as length falls, and a formula for the number of tokens a detector needs to reach a target true-positive rate. We calibrate thresholds within length bins with conformal p-values, and we add a bin-level rule that abstains when the detector cannot be informative at that length. On synthetic scores the formulas agree with simulation, and binned calibration holds the false-positive rate at its nominal value at every length. On three zero-shot detectors (log-likelihood, log-rank, Binoculars), three corpora and two open-weight generators, binned calibration gives a mean false-positive rate of 5.0% in every detector-length cell from 10 to 200 tokens, averaged over 300 random document-level resplits. A single threshold set at 200 tokens gives 29% for Binoculars at 10 tokens and about 1% for log-likelihood, as the closed form predicts. No detector reaches an 80% true-positive rate, and the best, Binoculars at 200 tokens, reaches 59%, so the rule abstains in every cell at that floor and releases only the 200-token bins at floors of 0.3 and 0.5.
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：Binoculars、AI生成文本检测、conformal prediction、误报率，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「arXiv cs.AI」（@Ritesh Janga, Justin Zhan），经 AITOP 多维评分管线结构化提炼。