新指标 Recall Fidelity Score 量化 LLM 记忆失真如何影响预测能力
Approximate Recall, Approximate Forecasts: Recall as a Diagnostic for LLM Forecasting Errors
这篇论文把 LLM 预测误差的来源拆到了记忆保真度上，还给出了可复现的 Recall Fidelity Score 指标。做模型评测或搭建预测类智能体的开发者可以拿来当作诊断工具，比单纯看预测市场得分更有解释力。
一篇 arXiv 论文提出 LLM 存在“近似召回”现象：模型能记住过去事件的粗略关联，却丢失具体细节，例如能答对 Anna Sawai 获得第 76 届 Emmy 奖最佳剧情类女主角，却把事件日期从 9 月 15 日错说成 9 月 22 日。作者设计了新指标 Recall Fidelity Score，用来衡量模型内部知识的不精确程度。论文核心问题是这种记忆保真度损失是否会传播到模型对未来事件的预测能力上。
在已结算的预测市场数据上，实验显示：模型对语义相似的历史事件召回保真度越高，在已结算市场上的预测误差越低，同一信号也能预测模型报价与开放市场价格的距离更近。这把预测市场基准评估从只看结果分数，推进到解释误差来源的机制层面。
Approximate Recall, Approximate Forecasts: Recall as a Diagnostic for LLM Forecasting Errors
Large Language Models (LLMs) exhibit approximate recall: they retain coarse associations about past events while losing fidelity on specific details. Past research on LLMs has quantified different models' forecasting ability in real-world events using prediction markets, but has not studied how losing said fidelity impacts their forecasting abilities. For example: while an LLM correctly recalls that the answer to: ''Did Anna Sawai win the Best Drama Actress in the 76th Emmy Awards?'' is ''Yes,'' it incorrectly states the event date as September 22 rather than September 15. In this paper, we introduce a novel metric called the Recall Fidelity Score to study how imprecision in internalized knowledge propagates to forecasting capability on future events. On resolved historical markets, we find that higher local recall fidelity around semantically similar past events predicts lower forecasting error on resolved markets, and the same signal predicts smaller distance from open markets prices.
本条情报归属 AITOP「论文」核心栏目，重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：LLM、预测市场、Recall Fidelity Score、评测基准，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「arXiv cs.LG」（@Prashanth Bhaskara, Shubhaankar Gupta, Seojoon Yeon），经 AITOP 多维评分管线结构化提炼。
