研究审计 PIT 智能体的未来记忆使用：Qwen 与 DeepSeek 均召回截断后事实
Auditing Future-Memory Use in Point-in-Time Language Agents
一篇挺有意思的审计研究：Qwen 和 DeepSeek 都记得截止日期后的公司收入，但做检索决策时反而不偏向记得多的那家，机制还没搞清楚。
arXiv 论文审计了 point-in-time (PIT) 智能体在 48 个历史公司对比任务中的行为。Qwen 和 DeepSeek 都能在闭卷测试中召回训练截止日期之后的公司收入等未来事实。但在预先指定的方向假设下，两个模型的检索行为并未偏向记忆更好的公司：Qwen 在 28 个不等召回任务上的四个分配估计为负，DeepSeek 在 20 个任务上的四项指标区间均低于零。作者建议除检查外部证据的时间截断外，还需分别评估记忆可达性、检索分配与最终输出。
Auditing Future-Memory Use in Point-in-Time Language Agents
Point-in-time (PIT) agents can recall post-cutoff facts even when their external evidence respects a historical cutoff. We examine whether this access predicts evidence gathering in 48 historical company-comparison tasks. Independent closed-book revenue probes identify the better-remembered company for each model on tasks with unequal recall, fixing the predicted direction before agent runs. We then compare paired runs with real or anonymized company identities while keeping external evidence PIT-valid. Both Qwen and DeepSeek recalled future facts, but neither supported the prespecified shift toward the better-remembered company. Qwen's four allocation estimates on 28 unequal-recall tasks were negative, with descriptive intervals spanning zero. A prospective DeepSeek replication showed an inverse pattern across all four metrics on 20 such tasks, with intervals below zero. These estimates remained negative after each company, pair, and future-fact omission, and all four post-hoc max-T-adjusted sign-flip p-values were below .05. In a prospectively specified follow-up, explicit future facts produced small estimated shifts away from the supplied-fact company, with intervals spanning zero, while final choices shifted toward it. The mechanism behind the natural inverse pattern remains unresolved. These findings support separate assessments of memory accessibility, search allocation, and final outputs alongside cutoff checks on external evidence.
