论文精选73°

GANDR:可验证法律答案生成中的声明审核

GANDR: Claim Auditing for Verifiable Legal Answer Generation

精选理由

GANDR系统让法律AI答案的每个声明都可验证,准确率比现有系统高11个百分点,代码已开源。

GANDR是一个双智能体系统,由起草者以结构化法律推理格式撰写答案,审核者独立验证每个声明与引用来源的匹配度。在185项法律基准测试中,GANDR达到70.8%的严格准确率,领先最强基线11.3个百分点。系统采用协议锚定提交规则,移除后准确率下降22.7个百分点。审核者对不充分声明的二分类检测F1值为0.84。

原文 · arXiv cs.AI

GANDR: Claim Auditing for Verifiable Legal Answer Generation

In high-stakes domains such as legal practice, a language-model answer is only useful to the extent that a reader can verify each claim against the source the system cites. Current grounded-generation pipelines score the answer as a whole, so a correct conclusion can rest on fabricated or loosely matched citations and still score well. Closing this gap requires both a system built for per-claim verification and an evaluation that measures it. We introduce GANDR (Grounded ANswer DRafter), a two-agent system in which a Drafter writes an answer in a structured legal-reasoning format and a separate Critic, with the same view as a human verifier, audits each claim against its cited source and emits a per-claim audit trace on every round. We pair it with a strict correctness criterion requiring every citation to resolve to a passage the retriever returned. On a 185-item legal benchmark where all six systems share one backbone, one retrieval surface, and one citation instruction, GANDR ranks first on every primary metric, reaching 70.8% strict accuracy and leading the strongest baseline by 11.3 points (p<0.01). Reverting the protocol-anchored commit rule lowers strict accuracy by 22.7 points, and the strict lead stays positive on three further backbones, at +3.2 to +6.5 points. This lead traces to the Drafter configuration and the protocol-anchored commit, not to rewriting. Against two law-trained annotators the audit flags under-supported claims at F1 0.84 as a binary detector, while its four-way verdict labels agree only weakly and are advisory. Code is available upon request.