医学LLM可验证推理综述:智能体级联错误与神经符号约束
Verifiable Logic in Medical LLMs: A Critical Review of Agentic Clinical Reasoning Optimization and Neuro-Symbolic Grounding
这篇综述把临床AI的幻觉问题拆得很细:多智能体系统会放大早期错误,符号逻辑层能当否决器。做医疗AI或关心监管认证的人该看看,它提出的推理轨迹审计思路可能影响落地标准。
arXiv 上的一项批判性综述(编号 2611.25898)指出,临床大语言模型的评估只看最终答案是否正确,而流畅但推理无效的输出、幻觉、因果无效推断和人口统计学捷径都能通过静态基准检测。综述梳理了四个范式:临床幻觉的产生机制、多智能体与工具增强系统中的智能体幻觉级联、过程奖励模型的步骤级监督、以及用形式逻辑引擎、本体推理器和知识图谱对神经输出施加可验证约束的神经符号接地。带符号否决层的系统在其编码领域内能提供相对保证,并在自用的安全指标上报告较大收益,但这些结果来自单项研究,多数在模拟或小样本环境中完成,无法直接与神经基线对比。
作者提出一个概念框架,把每种智能体失败模式映射到一个符号干预点,认为临床 AI 安全在很大程度上取决于系统架构。由于现行监管强调性能指标、不要求对推理步骤做验证,综述主张负责任的认证应在静态基准准确率之外,同步审计推理轨迹。
Verifiable Logic in Medical LLMs: A Critical Review of Agentic Clinical Reasoning Optimization and Neuro-Symbolic Grounding
Clinical large language models are usually judged by whether the final answer is right. Yet a model can reach a fluent, plausible output through invalid reasoning that static benchmarks do not detect, and confident hallucination, causally invalid inference, and spurious demographic shortcuts can pass conventional evaluation. This critical review covers four paradigms in turn: how clinical hallucination arises and why static metrics miss it; the agentic hallucination cascade, in which multi-agent and tool-augmented systems propagate and amplify early reasoning errors; process reward modeling as step-level supervision; and neuro-symbolic grounding, in which formal logic engines, ontology reasoners, and knowledge graphs place verifiable constraints on neural output. Within their encoded domains, systems with symbolic veto layers offer guarantees that hold relative to their encoded knowledge, and they report large gains on the safety measures they use. These results come from single studies, mostly in simulated or small-sample settings, and are not directly comparable with neural baselines. Process reward models remain bounded by the statistics of their training data. We propose a conceptual framework that maps each agentic failure mode to a symbolic intervention point, and we argue that clinical AI safety depends substantially on system architecture. Because current regulation emphasizes performance and does not require verification of reasoning steps, we argue that responsible certification should also audit reasoning trajectories alongside accuracy on static benchmarks.
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:临床AI、LLM、神经符号、幻觉,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「arXiv cs.LG」(@Youssef ElNahas, Adel Ammar, Soumaya Chaffar, Wadii Boulila),经 AITOP 多维评分管线结构化提炼。