OneCodeReviewBench 发布：用类型化决策评测代码审查能力
OneCodeReviewBench: Typed Decisions for Code Review and Search
做代码审查工具或想测模型审查能力的话可以看看，1968 个带明确契约的案例，OpenAI Decisions 拿了 81.3% 平衡准确率。
OneCodeReviewBench 是一个代码审查基准，包含 1,968 个合成案例，覆盖 12 种编程语言和 YAML，答案形式为布尔、分类和有序三种类型。基准在 OpenReview 上比较了 10 个模型-适配器系统，OpenAI Decisions 达到 81.3% 的布尔平衡准确率，Kev-27B 达到 76.8% 的分类准确率和 0.367 的评分平均绝对误差。基准还附带 586 个原始符号搜索案例和 40 个受限 Python 导入解析案例，解析器在其支持范围内匹配了全部 40 个参考答案。论文保留了 85 个与工程师答案一致的案例，供版本化工件复现分析。仓库级迁移和延迟/成本对比评测仍属未来工作。
OneCodeReviewBench: Typed Decisions for Code Review and Search
Code review requires finding functional bugs, memory-safety errors, security weaknesses, and performance problems across languages and software stacks. Tools can express bounded parts of this work as decisions about supplied code. We present OneCodeReviewBench, 1,968 synthetic cases across 12 programming languages and YAML, with Boolean, categorical, and ordered answers. Explicit contracts specify runtime assumptions, source scope, and answer criteria; separate evidence records support reference review. One author reviewed 150 cases; two additional engineers reviewed 100 of those cases. We compare ten model--adapter systems on the same cases and contracts: OpenAI Decisions reaches 81.3% Boolean balanced accuracy; Kev-27B reaches 76.8% Choice accuracy and Score mean absolute error of 0.367. Excluding two families with incomplete contracts preserves these leaders. Additional comparisons cover 586 cases with raw symbol search and 40 with a bounded Python import resolver; the resolver matches all 40 references within its supported profile. A versioned artifact reproduces these analyses and retains the 85 cases with matching engineer answers; the reported review subset excludes two incomplete-contract cases. Remaining reviewer disagreements and repaired inputs require resolution; repository-scale transfer and matched latency/cost evaluation remain future work.