Post-Audit Mirage:离线审计无法识别智能体部署后安全性的理论边界
Post-Audit Mirage: Identifiability Limits of Offline Agent Verification
一篇讲智能体安全审计盲区的论文:两个更新离线看起来一样,上线后果可能完全相反,作者还做了配套 benchmark 和监测工具。
arXiv 论文提出 post-audit mirage 问题:离线测试中安全与有害的智能体更新可以产生完全相同的可验证证据,部署后真实结果却相反。作者构造了 Post-Audit Mirage benchmark,覆盖 authorization、scheduling 和 shared-capacity 三类环境,并给出错误与弃判之间的权衡定理。配套的 IdentifiedRangeMonitor 基于 fresh group-labelled harm outcomes 输出 deploy、hold、cannot determine 或 unsupported 四种判定。实验显示,在有利设置下监测可恢复离线测试缺失的区分,但对稀有群体、小效应或缺失观测仍可能无法判定。
Post-Audit Mirage: Identifiability Limits of Offline Agent Verification
Offline tests can fail to reveal whether an agent update will remain safe after deployment. We operationalize this post-audit mirage using matched safe and harmful lifecycles that expose identical verifier-visible evidence while an isolated scorer records opposite deployment truth. Our theorem applies standard observational-equivalence and indistinguishability ideas to this executable audit boundary and states the resulting error-abstention tradeoff. The Post-Audit Mirage benchmark makes that boundary executable across authorization, scheduling, and shared-capacity environments. IdentifiedRangeMonitor uses fresh group-labelled harm outcomes to return deploy, hold, cannot determine, or unsupported. Experiments show that monitoring can recover the missing distinction in favorable settings but may remain undecided for rare groups, small effects, or missing observations. Verifier-agent evaluation must therefore test both calculation validity and whether the available evidence supports the deployment decision.