机制可解释性综述:理解模型内部能否让 AI 更安全
Understanding AI from the Inside: The State of Mechanistic Interpretability for AI Safety
一篇机制可解释性的全景综述,从 features、circuits 到 steering 都讲到了,还附代码 notebook 能自己跑实验,想做模型内部分析的不容错过。
arXiv 发布长篇综述《Understanding AI from the Inside》,系统梳理机制可解释性在 AI 安全中的应用。文章涵盖特征提取、多义性、Transformer 计算与 circuits 等基础概念,并介绍 circuit discovery、因果验证、训练数据归因等具体方法。作者提出统一框架来评估机制可解释性声明的证据强度,并划分六种研究方向。文中还讨论知识编辑、steering、监控等安全应用,同时指出该领域的局限与失败模式。配套发布的代码仓库提供可复现关键发现的 notebook。
Understanding AI from the Inside: The State of Mechanistic Interpretability for AI Safety
Neural networks are likely to be among the most consequential artefacts in human history. As these systems become more capable and autonomous, their potential to cause societal harm also grows, making AI safety one of the defining challenges of our time. Yet we continue to build increasingly powerful systems without fully understanding how they work, how they will behave in unfamiliar situations, or how to reliably control them. Mechanistic interpretability offers a path towards understanding these systems by studying their internal representations and computations, with the aim of evaluating, predicting, and controlling behaviour. We survey mechanistic interpretability for AI safety in order to answer the question: Does understanding how models work actually make them safer? We place mechanistic interpretability in the broader context of interpretability research and compare its mechanistic evidence with behavioural evaluation. We introduce the foundations of mechanistic interpretability, including features, polysemanticity, transformer computation, and circuits, and survey methods for feature extraction and analysis, circuit discovery and causal verification, explaining intermediate representations in natural language, learning dynamics, and training-data attribution. Beyond surveying existing methods, we propose a unified framework for evaluating mechanistic interpretability claims and characterise six research orientations by their goals, assumptions, and trade-offs. Finally, we examine how mechanistic understanding can support AI safety, including knowledge editing, steering, and monitoring, and assess its limitations, failure modes, open problems, and relationship to behavioural safety measures. We also release a repository of code notebooks with worked examples that reproduce key findings from surveyed methods, allowing readers to build intuition through hands-on experimentation.