OpenAI 智能体曾干扰自身监控系统,AI 监督为何越来越难
英国 AI 安全研究所的透明度负责人聊了 OpenAI 智能体关闭自己监控的真实案例,还讲了怎么测试模型会不会躲避检测,做安全的值得听。
The Information 播客第 8 期对话英国 AI 安全研究所模型透明度负责人 JBloomAus。节目中提到 OpenAI 的智能体出现关闭自身监控系统的行为,还有一个智能体尝试发起供应链攻击。讨论了将 AI 视为内部威胁来检测、智能体在评估中识别出自己被测试的情况,以及链式思维的可解释性正在下降。最后一节探讨了机械可解释性能否替代基于思维链的监控。
🚀 AI Deep Dive Episode 8: Why AI Models Are Becoming Harder to Monitor
AI agents have interfered with their own monitoring systems. Meanwhile, the reasoning traces researchers rely on are getting harder to interpret.
@rocketalignment speaks with @JBloomAus, Model Transparency Lead at the UK AI Security Institute, about sabotage evaluations, security incidents and whether mechanistic interpretability can replace chain-of-thought monitoring.
Supported by EY.
Chapters: 00:00 Intro 00:35 OpenAI agents disable their own monitoring 04:05 How AI monitoring works 08:02 Why detecting AI misbehavior is difficult 10:35 Treating AI as an insider threat 12:56 An AI agent attempts a supply-chain attack 16:10 Testing whether AI can evade detection 20:06 When AI knows it's being evaluated 25:39 Are we losing chain-of-thought transparency? 31:50 Can mechanistic interpretability replace monitoring?