新基准测试智能体能否胜任科研任务
精选理由
有人做了个 50 多个任务、覆盖 9 个学科的基准，专门测智能体能不能真正做科研，还带排行榜可以看谁家 agent 最能打。
一个新基准专门评估智能体做科学研究的能力，包含 50 多个任务，覆盖 9 个学科。基准附带排行榜，用来对比不同智能体的表现。评估标准包括选择正确方法、运行计算、从失败中学习，以及产出可验证的结果。
原文 · Ate-a-Pi
Can agents really do scientific research?
This is a new benchmark that focuses on this.
It comes with 50+ tasks across 9 scientific disciplines. There's also a leaderboard to see which agents work best.
Here is how we know an agent is good for scientific research:
• It should be able to choose the right methods • It should be able to run computations • It should be able to learn from failures • It should be able to produce verifiable results
