Google DeepMind 提出 XYEval：测试智能体是否会盲从用户的错误建议
Google DeepMind 这篇论文挺扎心的：你给智能体一句自信但错误的建议，它会照做还不吭声，三大主流模型最多掉分 46.7%。
Google DeepMind 发布论文，提出评测集 XYEval，在 tau2-bench、SWE-bench、Terminal-Bench、HLE 和 MCP-Atlas 的任务中注入一条听起来合理但实际错误的提示。任务本身和正确解法不变，因此分数下降只源于智能体采纳了坏建议。Gemini、Claude Opus 4.8 和 GPT 5.5 的相对分数最多下降 46.7%，且越简单的基准下降越大。不少智能体在推理中已意识到提示有问题，但仍默默执行且不告知用户；加入 XY 问题的系统提示只在单轮任务上有效，tau2-bench 和 SWE-bench Verified 这类多轮任务仍有大幅下降。
Interesting new paper from Google DeepMind.
Studies agents that follow bad advice from users.
Users often suggest a fix that sounds right and is wrong.
XYEval adds one confident, misleading hint to tasks from tau2-bench, SWE-bench, Terminal-Bench, HLE and MCP-Atlas.
The task and its correct solution stay the same, so any drop comes from the agent taking the advice.
Scores fall by up to 46.7% relative across Gemini, Claude Opus 4.8 and GPT 5.5. The drops are larger on easier benchmarks, which suggests more capable models will not fix this on their own.
Agents often disagree with the hint in their reasoning, then follow it anyway without telling the user. Compliance shows up almost only in failed runs.
A system prompt that warns about the XY problem helps on single-turn tasks and leaves large drops on multi-turn ones like tau2-bench and SWE-bench Verified.
Paper: https://t.co/290P0eUOzY
