论文精选

微软研究院与 Georgia Tech 发布 MANDATE:智能体失败分类法

精选理由

微软和 Georgia Tech 翻了 16 万条 Reddit 帖子加上 73 人访谈,整理出 agent 的 23 种失败模式,做 agent 产品的可以对照排查。

Microsoft Research 与 Georgia Tech 分析了 163,487 条关于 AI agents 的 Reddit 帖子,并调查了 73 位有经验的 agent 用户,梳理出委托型 agent 任务的出错方式。研究产出 MANDATE 分类法,将失败归为 7 组共 23 个类别,每类都描述了 agent 完成任务但方式超出用户预期的情况。用户评分显示,最严重的失败恰恰最难被察觉,而亲身踩过坑的人比只凭推测的人更容易发现问题。最常见的失败类型研究反而最少,例如 agent 持续运行却消耗时间与算力而无进展的情况,相关研究基础最薄弱。

原文 · DAIR.AI

Must-read paper from Microsoft Research and Georgia Tech.

This one has lots of great insights.

They used 163,487 Reddit posts about AI agents and surveyed 73 experienced agent users to map how delegated agent work goes wrong.

The result is MANDATE, a taxonomy of 23 failure categories in seven groups. Each describes an agent that finishes the job in a way the user didn't expect.

Users rated the most severe failures as the hardest to notice.

People who had already run into a failure found it easier to spot than people who only thought it was possible.

The most common failures are often the least studied.

Agents that keep running and spend time and compute without making progress are among the most commonly reported failures, and they have the thinnest research base.

https://t.co/UCgw7HZYfo

Chat with Paper: https://t.co/UDWpuOHsvu