SWE-Prometheus：用 60 个代码仓库评测工程治理能力的新基准

10 个模型在 22 个仓库的公开子集上评测，平均 Normalized Governance Improvement 介于 0.0568 到 0.

5760，行为破坏率在 0% 到 23% 之间。一个与仓库无关的模板在冻结的 10 仓库批次上取得平均 NGI 0.

272，但增益集中在 Tests & CI、Quality Gates 和 Documentation，在 Reproducible Environment 与 Dependency & Security 上没有提升。论文还报告了 no-op 条件的中位 NGI 为零，两名教师在 60 个维度评分中 exact agreement 达 57 个，并指出包含行为失败的对比中 Kimi-K3 占优。

SWE-Prometheus: Measuring Engineering Governance Improvements in Real-World Repositories

Large language model based coding agents have made substantial progress on repository-level software engineering tasks. Existing repository benchmarks, however, usually start from a human-identified issue and evaluate whether a patch satisfies a functional signal. We present SWE-Prometheus, a benchmark for the broader task of improving repository engineering governance. Each task provides a fixed snapshot and an open-ended objective, requiring the agent to identify risks, prioritize interventions, and verify the resulting changes. SWE-Prometheus evaluates six governance dimensions through paired evidence, clean-environment probes, behavior gates, and two independent teacher ratings of the same evidence. The benchmark contains 60 repositories; ten models are evaluated on a shared 22-repository public subset, where mean Normalized Governance Improvement ranges from 0.0568 to 0.5760 and observed behavior-breakage rates range from 0% to 23%. On a frozen ten-repository batch, a repository-blind template obtains mean NGI 0.272, but its gains concentrate in Tests & CI, Quality Gates, and Documentation; it improves Reproducible Environment and Dependency & Security on none of the repositories. This baseline makes the distinction between adding governance artifacts and producing execution-backed improvements measurable. The no-op condition has median NGI zero and standard deviation 0.073; two teachers agree exactly on 57 of 60 dimension scores for the same no-op evidence. For the two highest conditional-mean systems, common-valid NGI is similar, while full-pool comparisons that include behavior failures favor Kimi-K3. These results show why repository-governance evaluation should report improvement, behavior preservation, evidence quality, and coverage together.

本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」「各国 AI 监管法案、合规治理与主权算力政策」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：SWE-Prometheus、Kimi-K3、编码智能体、基准评测，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明：本文原始线索采编自「arXiv cs.AI」（@Jiajun Wu, Leixin Sun, Zihan Tan, Yitao Liu, Shuo Li, Jiaru Qian, Shanghaoran Quan, Chuangxin Zhao, Yangxu Liao, Yang Liu, Bin Chong, Guancheng Wan），经 AITOP 多维评分管线结构化提炼。截至目前，全网已有 6 家独立信源对该事件进行交叉印证与跟进报道（含 pandaily、vLLM、Fireworks AI 等），属于近期高关注度行业信号。