Singularity-Bench 发布：测试智能体能否端到端构建 AI 系统
Singularity-Bench: Can Agents Build Agents?
这个基准让智能体从零写训练、推理、编排系统，再和人类基线硬碰硬。结果挺扎眼：Opus 5.5 配 Claude Code 六项全过人类线，其他智能体平均只有 0.58。做 agent 评测或基础设施的工程师都该看看。
研究人员推出 Singularity-Bench，一个开放式基准，涵盖预训练、中期训练、后训练、推理、agent harness 和集群编排六项端到端任务，要求智能体从最小起点实现核心 AI 系统并与人类工程基线在同硬件下对比。
基准还通过覆盖 OWASP Agentic Top 10 中 9 类攻击的红队测试，暴露了 Harbor 框架的奖励劫持漏洞，并推出能阻断全部测试攻击的扩展 Breakwater。在 11 个由 7 个前沿模型和 2 个 harness 组成的智能体中，Opus 5.
5 配合 Claude Code 在全部六项任务上达到或超过人类基线，Singularity-Index 达 1.25，其余智能体平均仅 0.
58。团队希望该基准追踪智能体在 AI 开发中的自动化程度，以及何时能从辅助转向独立构建更强的后继系统。
Singularity-Bench: Can Agents Build Agents?
Frontier AI agents increasingly succeed on realistic, long-horizon tasks and are beginning to improve the systems through which they operate. Yet existing benchmarks often restrict agents to narrow action spaces, use simplified environments, or rely on verification methods that are unreliable for evaluating arbitrary agent-generated systems. We introduce Singularity-Bench, an open-ended benchmark for autonomous AI development with six end-to-end tasks spanning pre-training, mid-training, post-training, inference, the agent harness, and cluster orchestration. Each task requires agents to implement core systems from a minimal starting point and evaluates them against mature human-engineered baselines on identical hardware. Additionally, through extensive red-teaming covering 9 of the 10 categories in the OWASP Top 10 for Agentic Applications, we expose reward-hacking vulnerabilities in the conventional Harbor framework and develop Breakwater, an extension of Harbor that blocks all attacks tested in our study. Across 11 agents built from 7 frontier models and 2 harnesses, Opus 5.5 with Claude Code matches or outperforms the human baseline on all six tasks with a Singularity-Index of 1.25, suggesting it may have the skills needed to reach the singularity. The remaining agents average only 0.58. Our vision is for Singularity-Bench to track how much of AI development agents can already automate and when they may transition from assisting to taking end-to-end responsibility for building stronger successors.
本条情报归属 AITOP「模型」核心栏目，重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」「中美大模型技术代差与商业化对照」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：Claude Code、Singularity-Bench、benchmark、智能体，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「arXiv cs.AI」（@Yang Zhou, Krish Agarwal, Zhuoming Chen, Zerui Wang, Zhenyu Zhang, Wenshuang Li, Xinpeng Wei, Zhichen Zeng, Jingyu Liu, Jue Wang, Sen Wu, Ziheng Jiang, Zhen Dong, Han Cai, Binhang Yuan, Ying Sheng, Ce Zhang, Bo Li, Beidi Chen），经 AITOP 多维评分管线结构化提炼。
