研究:Web 研究型智能体如何解读用户优先级
How Web Research Agents Interpret User Priorities
同一句"重点关注增长",OpenAI 系智能体倾向 Microsoft,Anthropic 系却选 Alphabet。这篇论文拆开了 36 次实验里两家的评判标准差异,做智能体评测或用它们做研究的人很该看看。
arXiv 论文(编号 2611.22989)考察了基于 OpenAI 与 Anthropic 模型的两个 Web 研究型智能体如何理解用户提出的"优先级"指令。实验共 36 次运行,任务为对比 Microsoft 与 Alphabet 两家公司,提示词分别强调增长、韧性或不设优先级。在 24 份带优先级的报告中,21 份确实围绕该优先级展开,但评判标准各不相同:增长优先级下,OpenAI 系报告把风险与可预测性纳入考量、更倾向 Microsoft,Anthropic 系报告按增长速度评判、更倾向 Alphabet;韧性优先级下,多数报告认为 Microsoft 更有韧性。
研究者还分析了报告的搜索记录、返回来源与引用来源,发现测试的搜索与来源指标中,只有 Anthropic 智能体的搜索措辞随条件出现可检测差异,OpenAI 系则无明显变化。这说明同一句用户指令在不同模型里会落到不同的评判框架上,报告的关注点一致不代表结论逻辑一致。
论文据此提出,对研究型智能体的评测应检查报告背后的评判标准,而非只看聚焦度或事实准确性;产品层面可以让用户查看并调整这些标准。对金融分析等高风险场景的使用者,这意味着同样一句"重点关注增长"可能得到方向相反的推荐。
How Web Research Agents Interpret User Priorities
Web research agents gather information online and write cited reports, including financial analysis. Users can state a priority to guide this research, but a priority may not say how to judge the options, leaving the agent to choose the criteria. We examine how two such agents, built on OpenAI and Anthropic models, interpret a stated priority. In 36 runs, the agents compared Microsoft and Alphabet as businesses. Prompts asked for special attention to growth or resilience, or gave no priority. We analyzed reports, searches, returned sources and cited sources. Of the 24 reports given a priority, 21 focused on it, but their criteria varied. Under the growth priority, OpenAI reports that ranked the companies accounted for risk or predictability and favored Microsoft, while the Anthropic ones judged growth by its pace and favored Alphabet. Under the resilience priority, reports used partly different criteria, and most named Microsoft as the more resilient company. Of the search and source measures tested, only the Anthropic agent's search wording differed detectably by condition. These findings suggest evaluations should examine the criteria behind a report's judgements, not only its focus or accuracy. Interfaces could let users see and adjust those criteria.
本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:OpenAI、Anthropic、智能体、AI评测,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始信息发布自一手权威信源「arXiv: Anthropic」(Tier-1 一手发布),具备高确定性参考价值。