FinVeriQA 开源多智能体金融问答系统:小模型路由、Python 计算、MCP 工具链
FinVeriQA: Financial Research Agents for Question Answering: Routing, Retrieval, MCP Tool Use, and Provenance-Aware Evidence Verification
这篇把金融问答 Agent 拆成路由、计算、检索、写作、校验五步,用数字说话:小模型路由反而赢了大模型,Mistral-7B 算数只有 29% 正确率。做 RAG 或智能体的人该看看它的校验设计——证据自检靠不住。
开源多智能体原型 FinVeriQA 针对 SEC 文件和实时行情构建金融问答流水线:紧凑决策模型 Laya 负责问题路由并拦截投资建议请求,所有数值由 Python 计算,Mistral-7B 生成带引用的回答,代码校验器核对引用 ID 与计算结果,整条链路通过 MCP 服务器对外暴露。
实测数据显示,Laya 路由准确率 0.85,高于 Mistral-7B 的 0.52(总体 0.86 对 0.68,McNemar p = 0.0003);Mistral-7B 在 100 道金融计算题上只算对 29%,Python 则全对;接入实时报价把价格误差从 210% 降到 0.00%,混合检索在 50 道文件问题上取得 76% Hit@5。
但案例研究暴露了关键问题:两个智能体回答都通过了校验,却把 AMD 的样本数据标成 NVIDIA 和 Alphabet。在 FinRank 基准上的对照实验解释了原因——证据被污染时自一致性仍保持高置信(一致性 0.55-0.59,准确率从 0.50 跌到 0.00),共享证据校验只能抓到 35% 错误,对照原始段落核查可抓到 77%(p = 5.96×10⁻⁸);DeepSeek-R1-8B 为 68% 对 90%。结论是:校验必须独立于写作模型看到的那份证据。
FinVeriQA: Financial Research Agents for Question Answering: Routing, Retrieval, MCP Tool Use, and Provenance-Aware Evidence Verification
Large language models (LLMs) write fluent financial analyses but miscompute ratios, recall stale prices, and route questions to the wrong tool with high confidence. We ask which component should do each job in a financial question-answering assistant, and study it with FinVeriQA, an open-source multi-agent research prototype over SEC filings and live market data: a compact typed-decision model (Laya) routes each question and screens out advice requests, Python computes every number, retrieval and tools fetch evidence, Mistral-7B writes a cited answer, a code validator checks that cited IDs exist and that numbers match computed notes, and a Model Context Protocol (MCP) server exposes the pipeline. In live experiments, Laya routed better than Mistral-7B (0.85 vs. 0.52; overall 0.86 vs. 0.68, McNemar p = 0.0003); a 12B Mistral decided no better, and DeepSeek-R1-8B decided better (0.79 vs. 0.68, p = 0.0192) but about 9.9 times more slowly. Mistral-7B computed 29% of 100 financial calculations correctly against 100% for Python, a live quote cut price error from 210% to 0.00%, and hybrid retrieval reached 76% Hit@5 on 50 filing questions. Yet in a live case study, both agent answers passed validation while describing AMD sample data under NVIDIA's and Alphabet's names. A controlled upper-bound study on the FinRank benchmark explains why: when one figure in the shared evidence is corrupted, self-consistency stays confident (agreement 0.55 to 0.59 while accuracy falls from 0.50 to 0.00), and a shared-evidence check (the kind our validator performs) catches 35% of the errors, whereas a check against the original passage catches 77% ($p = 5.96 \times 10^{-8}$); for DeepSeek-R1-8B, 68% vs. 90%. Verification must be independent of the evidence the writer saw.
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:FinVeriQA、MCP/工具、Mistral-7B、DeepSeek-R1-8B,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始信息发布自一手权威信源「arXiv: DeepSeek」(Tier-1 一手发布),具备高确定性参考价值。截至目前,全网已有 11 家独立信源对该事件进行交叉印证与跟进报道(含 NVIDIA AI、IT之家、DAIR.AI 等),属于近期高关注度行业信号。