Sam Altman 转发 Nick Cammarata：可解释性研究进展超预期
Sam Altman 转发了一位 OpenAI 前可解释性负责人的长文，说现在万亿参数模型的神经元大多能看懂了，和他早年只解读出 2% 的判断完全反转。
OpenAI CEO Sam Altman 转发前 OpenAI 机制可解释性项目负责人 Nick Cammarata 的观点，认为可解释性可能是当前最重要的科学问题。Cammarata 曾参与 OpenAI 早期机制可解释性工作，当时持悲观态度，仅有约 2% 的神经元可被解读。他后来启动了 OpenAI 首个大规模自动化可解释性项目，如今发现直至数万亿参数规模的模型中绝大多数神经元均可解读，许多此前被视为“纠缠神经元”的案例其实被误读。他认为自动化科研工具的进步可能让可解释性问题很快得到解决，并上线了 goodfire.com 相关页面。
Very happy to see this from Nick! Nick @nickcammarata interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works already-clean.goodfire.com 🔗 View Quoted Tweet 💬 118 🔄 56 ❤️ 1105 👀 101276 📊 150 ⚡
