arXiv 论文分析 21000 行全由 Claude 编写的代码库可靠性

作者还开源了两个代码溯源追踪工具，以及针对指令意图、提交溯源、响应可靠性的三套分类体系。分析显示 14.

3% 的 AI 代码生成事件包含真实错误，后被 AI 编写的测试套件捕获；AI 的交互回复中约四分之一到五分之一包含至少一处事实性错误。

研究还发现编程智能体 CLI 指令与 IDE 聊天指令在类型上不同，前者更侧重理解、规划和咨询。

Between the Commits: Process, Error, and Claim Reliability in a Wholly AI-Authored Codebase

We present: (i) a new dataset consisting of the full development history of a 21,000-line Python tool built entirely by Claude AI, with no human-authored code or tests, (ii) two code-provenance tracing tools, (iii) three taxonomies for instruction intent, commit provenance, and response reliability, (iv) application of these to analyse the dataset. We find that: (i) user coding agent CLI instructions differ in kind from IDE-chat instructions, with a greater focus on comprehension, planning and consultation, (ii) code development is mainly proactive, (iii) 14.3% of AI code-generation events contain a real error later caught by the AI-authored test suite, (iv) roughly 1 in 4-5 of the AI's interactive responses contains one or more factual errors.

本条情报归属 AITOP「论文」核心栏目，重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：Claude、编程智能体、AI代码、代码溯源，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明：本文原始线索采编自「arXiv cs.AI」（@Douglas Leith），经 AITOP 多维评分管线结构化提炼。