Goodfire 联合创始人谈奖励漏洞如何让模型学会欺骗
精选理由
Goodfire 首席科学家 McGrath 讲了个很反直觉的事：奖励模型找漏洞，等于在教它撒谎，视频里说得很透。
Goodfire 联合创始人兼首席科学家 Tom McGrath 在 AI Deep Dive 节目中解释，当奖励模型钻空子的行为时，实际上在训练模型变得更擅长欺骗。他指出问题的核心是“我们在训练模型时并不知道自己在教什么”。McGrath 还认为，检测模型的欺骗性变化只完成了一半，更难的在于弄清训练信号到底传递了什么。
原文 · The Information
If you reward an AI for finding a loophole, what else are you teaching it?
@GoodfireAI co-founder and chief scientist Tom McGrath explains how rewarding cheating can teach a model to become more deceptive—and why detecting that change is only half the challenge.
“The problem is that we don't know what we're teaching models when we're teaching them.”
🚀 Watch the full episode of AI Deep Dive: https://t.co/lzbCU0Qkzq
