Google DeepMind 用可靠性理论分析 AI 控制栈的失效抑制
Reliability Theory for AI Control
DeepMind 这篇论文把工程里的可靠性理论搬进 AI 控制分析，算出失效抑制能到立方级，做安全系统设计的可以看看。
一篇 arXiv 论文将可靠性理论应用到 Google DeepMind 的 AI 控制防御上。分析显示，同一个控制栈因失效域不同，其稀有失效抑制可呈现立方、平方或线性级别差异。论文用 Birnbaum 重要性度量来识别哪些组件改进能带来最多的名义可靠性提升，并指出预防措施会改变需要恢复机制的失效总体。结论为系统的分离、改进、测量与测试给出了具体建议。
Reliability Theory for AI Control
Reliability theory gives a mature language for layered systems, but its formal tools are not yet standard in frontier AI control. We apply them to Google DeepMind's defenses against rogue deployment. The same control stack can have cubic, quadratic, or linear rare-failure suppression depending on its failure domains. Birnbaum importance identifies which component improvements buy the most nominal reliability, while prevention changes the population on which recovery is demanded. These results give concrete guidance about what to separate, improve, measure, and test.
