论文

每条边都需引用原文:用语言模型智能体构建医疗软件采用知识图谱

Every Edge Needs a Quote: Building a Quote-Grounded Knowledge Graph of Healthcare Software Adoption with Language-Model Agents

精选理由

医疗软件采购信息散落各处,这篇论文让智能体去爬并要求每条关系都附原文引用,61 家厂商测出 0.94 精确率,做知识抽取的可以看看这套防幻觉设计。

arXiv 论文提出一套 pipeline,从公开网页提取医疗软件的客户与集成关系,构建知识图谱,规则是每条边必须附带来源页面的逐字引用。系统用确定性爬取加预算受限的 ReAct 智能体搜集页面,由宽松抽取器提出候选关系,再经分层 grounding 阶段筛选,包括读取整页的 referent 步骤和 5 次模型调用对 9 项标准投票。在 61 家厂商上,pipeline 对页面级标注达到 0.94 精确率和 0.71 召回率;基于投票的置信度分数在 0.90 精确率下达到 0.90 召回率。论文还讨论了窄范围模型调用、字符串匹配证据与投票机制如何应对长上下文退化和多次运行间的方差。

原文 · arXiv cs.AI

Every Edge Needs a Quote: Building a Quote-Grounded Knowledge Graph of Healthcare Software Adoption with Language-Model Agents

Hospitals regularly buy software that clinicians and patients depend on. Because these purchases are costly, hard to reverse and can affect patient care, buyers look for evidence of which peer institutions already use a product and which systems it connects to. This evidence is public but scattered across vendor websites, press releases and third-party pages. Language-model agents can now collect it at low cost, but the correctness of the extracted claims is difficult to establish, and a wrongly listed customer can mislead buyers and misrepresent an organisation. We present a pipeline that builds a knowledge graph of customer and integration relations from the open web under one rule: every edge must be supported by a verbatim quote from its source page. Pages come from a deterministic crawl of each vendor's site and a budgeted ReAct agent that searches the rest of the web. A permissive extractor proposes relations with quotes, and a layered grounding stage decides which to keep, using a referent step that reads the whole page when a quote names only one side, deterministic checks, and a panel of five model calls voting on nine criteria. On 61 vendors, the pipeline reaches 0.94 precision and 0.71 recall against a page-level key covering the 6,428 candidate relations proposed by its extractor, whose verdicts were drafted by one model and re-checked with a second. Against the golden reference, the marketplace's own curated list, it scores lower, but 89% of the relations it keeps that the reference lacks are stated on a page. A confidence score built from the panel's votes keeps more correct relations than the fixed rule at the same precision, and reaches 0.90 precision at 0.90 recall. We also discuss how narrow, bounded model calls, string-matched evidence and voting address long-context degradation and run-to-run variance.