Goodfire 联合创始人 Tom McGrath 谈可解释性:模型思考未必都写在推理里
Goodfire 的 Tom McGrath 聊可解释性,讲清了模型写出的推理和脑子里实际想的事可能不一样,还有怎么用内部信号抓模型作弊,做 AI 安全的别错过。
The Information 播客 AI Deep Dive 第 5 期邀请 GoodfireAI 联合创始人兼首席科学家 Tom McGrath,讨论模型写出的推理文本与内部真实计算之间的差距。McGrath 介绍了可解释性研究中的 AI 脑扫描方法,说明内部信号如何用于发现模型在基准测试中作弊。节目还梳理了研究者如何识别模型内部概念、逆向工程语言模型,以及在不掩盖问题的情况下修改模型行为的路径。
🚀 AI Deep Dive Episode 5: Can We See Inside an AI’s Mind?
AI can write down its reasoning. But what happens when more of its thinking never makes it onto the page?
@rocketalignment and @GoodfireAI co-founder and chief scientist Tom McGrath explore what AI “brain scans” can reveal beyond written reasoning, how internal signals could help catch cheating, and what it would take to understand—and change—what models learn.
00:00 Intro 00:39 Tom McGrath and AI brain scans 09:01 Written reasoning and hidden computation 17:45 Why AI is hard to understand 24:07 How researchers identify concepts 31:10 What interpretability could change 34:42 Reverse engineering a language model 43:09 The field’s approaches and investment 51:51 Changing models without hiding problems 1:00:10 AI agents studying AI