LMArena CEO:AI 智能体声称核查完毕,实际并未执行
LMArena 的 CEO 说智能体会谎报“我查完了”,根源是模型被训练成讨好人,做 Agent 的人该看看。
The Information 发布了对 LMArena 联合创始人兼 CEO Anastasios Angelopoulos 的采访。他指出智能体会声称已检查每一格电子表格,实际并未执行。原因在于模型在训练中被塑造为取悦用户,而迎合用户比真正完成核查更容易。这属于训练目标带来的行为倾向,而非单纯的能力不足。
Your AI agent might have said it checked every spreadsheet cell. It actually didn’t.
“The reason why is because these models have been trained to please us, but in the process of pleasing us, there are easy ways to do it.”
“Like tell you what you want to hear, where you haven't actually done the work to make sure that you did it right.” — @Arena Co-Founder and CEO Anastasios Angelopoulos