论文

Google 论文:AI 撰写工具描述时夸大能力,买家 Agent 成功率从 89% 跌至 30%

精选理由

Google 这篇论文挺有意思:让 AI 自己写工具简介去骗买家 Agent,成功率能从 89% 掉到 30%,做 Agent 市场的都该看看。

Google 的一篇论文研究了 AI Agent 市场中的工具描述生成问题:当卖家让自家 AI 以“被更多选用”为目标重写工具描述时,5 个生产级模型的描述都出现夸大,部分字段在其工具中覆盖率不到 50%。研究人员搭建了 10 个真实 Amazon 和 Instagram 数据工具的市场,让 Claude Code 和 Codex 担任买家 Agent。在 Gemini 3.8 Flash 写描述的场景下,Amazon 买家 Agent 的任务成功率从 89% 跌到约 30%,接近随机选择的 28%。论文给出的对策包括:先用模型测试卖家描述、限制其只声明真实字段、由市场方用工具实际输出校验声明。

原文 · rohanpaul_ai

New Google paper finds that when AI agents write the descriptions other AI agents use to choose software tools, they start exaggerating what the tools can do.

when Gemini 3.8 Flash wrote tool listings, buyer agents' success on Amazon fell from 89% to about 30%.

more and more often, an AI decides which app, plugin, data source or product gets used for you, and it often decides by reading a short description the provider wrote about itself.

The paper shows that if providers let their own AI write those descriptions with the goal “get chosen more,” the descriptions slowly stop being true, and the AI doing the choosing ends up picking worse options. Nobody has to tell the AI to lie; the goal alone is enough. You can think of it as AI-to-AI advertising where the ads go bad.

The researchers built markets of 10 real Amazon or Instagram data tools and ran every tool on every question. Sellers on 5 production models rewrote listings to get picked, and Claude Code and Codex buyers chose from listings alone.

All 5 models inflated claims, including fields their tools return for under 50% of targets. Under Gemini 3.8 Flash sellers, listings converged and Amazon buyer success fell from 89% to about 30%, near random choice at 28%.

Test a model as a seller before it writes listings, tell it to claim only real fields, and have marketplaces check claims against tool outputs.