论文

论文:文本相似度护栏难以识别多智能体间的注入传播

Safe Resemblance, Silent Propagation: Limits of Text-Similarity Guardrails for AI Agent Safety

精选理由

做智能体安全或内容过滤的工程师应该看看这篇。它用三智能体邮件接力实测 DonkeyRail 分类器,漏检注入传播又把引用复用全误报,还找出了 21 个已发布配置里的 ROC 操作点错误。如果你在靠文本相似度做防线,这篇能帮你少踩坑。

arXiv 论文(编号 2611.21520)评估了 DonkeyRail——一个结合 BLEU、ROUGE-L 和 METEOR 特征的公开文本相似度分类器——在三智能体邮件接力场景中能否拦截注入指令的传播。

正文

在默认阈值下,19 个功能性输出中漏检 8 个,同时在 192 条合法回复中误报 70 条;所有完整引用和模板复用类回复全部触发误报。

针对某一模板集校准低误报率后,新阈值只能检出 1 个功能性输出,且在新模板上仍产生 15 次误报,说明校准无法跨模板迁移。论文还指出已发表的 21 个分类器配置存在操作点错误:ROC 插值被用来替代直接阈值应用;

并从理论上证明,针对表层文本标准的高 AUC 并不构成对行为目标的分布无关保证,而现有评估标签缺少估计这一差距所需的联合观测。授权分析进一步表明,相同的可见对话记录在不同治理策略下可能需要不同的安全判断,这是仅靠观测的分类器无法解决的。

作者主张用行为层面和策略驱动的评估来取代文本相似度护栏。

原文 · arXiv cs.AI

Safe Resemblance, Silent Propagation: Limits of Text-Similarity Guardrails for AI Agent Safety

When AI agents relay information to one another, an output can preserve an injected instruction's effect on downstream agents even after its wording changes, yet legitimate quotation and template reuse can produce high textual overlap with the source, so text-similarity guardrails that attempt to flag the former risk conflating these two cases. We test this distinction using DonkeyRail, a public classifier combining BLEU, ROUGE-L, and METEOR features, deployed in a controlled three-agent email relay, defining a behavioral measure in which an output counts as functional (the injected procedure induces fresh computations in two subsequent agents) independent of the detector's score, alongside legitimate controls that explicitly authorize quotation and reuse. At its default threshold the classifier misses 8 of 19 functional outputs while flagging 70 of 192 legitimate replies, with every completed quotation and template-reuse reply triggering a false alert; calibrating for low false positives on one template set does not transfer, as the recalibrated threshold detects only one functional output and still produces 15 false alerts on new templates. We also identify an operating-point error in all 21 published classifier configurations, where ROC interpolation replaces direct threshold application, and show formally that high AUC against a surface-level text criterion provides no distribution-free guarantee of high AUC against the behavioral target, while the existing evaluation labels lack the joint observations needed to estimate the gap. An authorization analysis further shows that identical visible transcripts can require different safety decisions depending on the governing policy, a distinction no observation-only classifier can resolve. These results argue for behavioral and policy-grounded evaluation over text-similarity guardrails.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:DonkeyRail、智能体、AI安全、注入攻击,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Charles Xiao),经 AITOP 多维评分管线结构化提炼。