BioXArena-Pro 发布:评测自主生物医学机器学习智能体的研究级基准
BioXArena-Pro: a research-grade benchmark for autonomous biomedical machine learning agents
做科研自动化或 Agent 评测的可以看看这个新基准,64 个真实生物医学建模任务,防泄漏设计做得比较扎实。结论挺有意思:Agent 强在灵活改方法,但基础模型被禁用就掉 9 分。想测自家 Agent 在科研场景上限的团队直接能用。
BioXArena-Pro 是一个面向生物医学机器学习智能体的研究级基准,包含 64 个研究级建模任务,覆盖 8 个生物医学领域,每个任务都从一篇已发表研究重建而来,多数任务对整个生物单元(如物种或队列)做留出,防止信息泄漏。
在 22 种智能体配置的评测中,最佳配置得分为 55.1/100,且智能体框架(scaffold)的影响几乎与底层模型相当。
团队为每个任务重新运行了原论文的已发表方法:在原方法可直接套用的任务上,最佳智能体与方法打平;在需要重建的任务上,方法落后智能体 8.
2 分,说明智能体的优势在于对新数据的自适应调整。预训练模型(包括生物基础模型)也起到关键作用:禁止使用后,最强智能体在这些任务上得分下降 9 分。
所有任务经过多轮 AI 辅助审计和最终人工复核。
BioXArena-Pro: a research-grade benchmark for autonomous biomedical machine learning agents
Coding agents are increasingly proposed as autonomous builders of biomedical models, but existing benchmarks are small, near saturation or open to leaked information. We present BioXArena-Pro, 64 research-level modelling tasks from eight biomedical domains, each rebuilt from a published study; most hold out whole biological units such as species or cohorts. Across 22 agent configurations, the best scores 55.1 of 100, and the scaffold matters almost as much as the model. We re-ran a published method for every task: where the published recipe applies unchanged, the best agent and the methods tie; on rebuilt tasks the methods trail by 8.2 points, suggesting that agents gain by adapting to new data. Pretrained models (including biological foundation models) matter where the strongest agent uses them: forbidding them decreases its score on those tasks by 9 points. Every task was audited in several AI-assisted rounds and a final human review.
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:BioXArena-Pro、生物医学、智能体评测、基准测试,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「arXiv cs.LG」(@Loka Li, Duzhen Zhang, Xingbo Du, Leonard Song, Kun Zhang, Le Song),经 AITOP 多维评分管线结构化提炼。