论文

多智能体系统 TriAgent 持续修复文档漂移,43 天分诊 499 张工单

Mind the Gap: A Deployed Multi-Agent System for Documentation Maintenance

精选理由

工程团队常遇到的文档跟不上代码的问题,这篇给了可跑的方案。TriAgent 从真实支持工单里自动挖文档缺口,修完后智能体重答同一问题,71.4% 的盲评案例答案更好。做智能体落地和文档运维的工程师可以直接抄架构。

论文介绍了一个已部署的多智能体文档维护系统,核心是名为 TriAgent 的分诊智能体,持续从已解决的支持工单和代码-文档差异中提取信号,定位并修复文档滞后。

正文

系统以 shadow 模式运行 43 天,分诊了 499 张工单,发现 92 处文档缺口。在由三个独立模型家族的 LLM 评审组成的小组评估中,TriAgent 可验证断言的正确率为 81.

1%(302 条断言中 245 条正确),在长度可比的成对比较中,评审小组在 70.5% 的对子上偏好 TriAgent 的回复,人类工程师的为 23.

2%。修复文档缺口后,智能体重答同一张工单,盲评评审在 71.

4% 的情况下(n=35)更偏好修复后的答案,证实了修复带来的实际改善。人类与智能体的回复往往互补:人工常挖掘出埋在文档内部或完全缺失的洞察,系统把每条洞察转化为文档缺口,经人工审核后变成修复。

随着用户越来越多依赖以文档为主要接口的 LLM 智能体,过时信息会被当作权威传播,这类自动化的文档维护流程对运营支持团队和文档工程团队有直接参考价值。

原文 · arXiv cs.AI

Mind the Gap: A Deployed Multi-Agent System for Documentation Maintenance

The "AI revolution" has enabled teams across the technology industry to deliver software features at unprecedented velocity. While code evolves faster than ever, customer-facing documentation often struggles to stay up to speed. As users increasingly rely on LLM-based agents that consume documentation as their primary interface, the impact of this documentation gap worsens: stale information propagates and is treated as authoritative. Signals for identifying where documentation has fallen behind already exist in abundance, both explicit (e.g., resolved support tickets revealing what documentation lacks) and implicit (e.g., code-documentation discrepancies detectable from automated loops). We deploy a multi-agent system, built around a triage agent we call "TriAgent," that continuously extracts these signals and repairs documentation drift. Over 43 days of shadow-mode operation, our system triaged 499 tickets and surfaced 92 gaps in the documentation. Using a panel of LLM judges from independent model families, 81.1% of TriAgent's verifiable assertions were correct (245 of the 302 assertions the human resolution addresses; majority of three judges); on length-comparable pairs the panel preferred TriAgent's response in 70.5% of pairs and the human engineer's in 23.2% (the rest had no judge majority). Importantly, agent- and human-sourced replies tended to be complementary, with the human often surfacing insights that were either buried within the existing documentation or missing from it entirely; our system turns each such insight into a documentation gap and, after human review, a fix. Repairing a gap measurably improved the agent's later answer: when the agent re-answered the same ticket after the repair, blinded judges preferred the post-repair answer over its own pre-repair answer 71.4% of the time (n=35). A blinded human audit confirms strong alignment on self-improvement.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:智能体、TriAgent、RAG、文档维护,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Lorenzo Masoero),经 AITOP 多维评分管线结构化提炼。