FDE-Gym 基准：测试智能体能否胜任一线交付工程师
有个新基准 FDE-Gym，拿 95 个真实交付项目考智能体，7 家厂商 13 种配置最好的也才解决 14 个，看看你的 Agent 差在哪。
FDE-Gym 用 95 个真实客户交付项目测试智能体，要求其像 Forward-Deployed Engineer 一样询问脚本化干系人、在类生产系统中工作、找出客户未言明的约束。交付效果按客户自身成功指标的改善程度打分，多余消息、试运行和等待时间会被扣分。在 7 家提供商的 13 种配置中，最佳配置完整解决了 95 个任务中的 14 个，最高平均分 0.56。主要失败原因是遗漏未言明的约束：即使在最优配置收集到全部关键信息的 23 个任务上，也只有 10 个达到 0.8 分以上。
// FDE-Gym: Agents as Forward-Deployed Engineers //
They found that the best agent configuration fully resolved 14 of 95 real customer-delivery tasks.
FDE-Gym tests agents as forward-deployed engineers. Each task comes from a real delivery project. Agents must question scripted stakeholders, work in production-like systems, and find constraints the customer never stated.
A delivery is scored by how much it improves the customer's own success metric, with penalties for extra messages, trial runs, and waiting time.
Across 13 configurations from seven providers, the top mean score is 0.56. The main failure is missing unstated constraints. Even on 23 tasks where the best configuration collected every key fact, it reached 0.8 or higher on only 10.
Highly recommended benchmark to look at.
https://t.co/EwGp9jsGae
Chat with Paper: https://t.co/c8w3kBWyW6
