ClaimWriter 重写深度研究报告流程,幻觉减少 2.6–4.5 倍
Salesforce 研究团队做了个 ClaimWriter,写深度报告前先逐条核对来源,幻觉少了 2.6–4.5 倍,代码和基准都开源了。
深度研究报告常出现流畅但缺事实、错引来源的问题。ClaimWriter 从来源中提取事实,映射到基于用户查询的大纲,再逐节从带来源链接的声明起草。团队在 Enterprise Deep Research、NVIDIA AI-Q、Open Deep Research 三个框架中只替换报告写作模块,幻觉减少 2.6–4.5 倍,必要事实召回提升 1.2–1.7 倍,整体质量基本保持。同时发布 ClaimProbe,逐条对照检索证据审计报告声明。
Redesigning and auditing deep research writing for faithful reports: https://t.co/kZXAiqs78z
Deep research agents can write fluent, well-scored reports that still miss key facts, cite the wrong sources, or make claims no source supports.
ClaimWriter changes how the report gets written. It extracts facts from sources, maps them to an outline built from the user's query, and drafts each section from source-linked claims.
We swapped only the report writer into three deep research frameworks (Enterprise Deep Research, NVIDIA AI-Q, and Open Deep Research), keeping search and evidence collection fixed: → 2.6–4.5× less hallucination → 1.2–1.7× better recall of necessary facts → Overall report quality largely preserved
To measure the difference, we built ClaimProbe, a claim-level audit that checks each report claim against the retrieved evidence.
Code and benchmark: https://t.co/kF5EFZ8FQq
#FutureOfAI #EnterpriseAI