研究:相同预测不代表相同信念更新,语言模型贝叶斯推断机制解析
Same Answer, Different Future: Matched Predictions Do Not Imply Matched Belief Updating in Language Models
三个开源模型上做的对照实验,量化了模型“学来的信念”到底值多少条证据,做可解释性的朋友可以看看。
论文设计了一个带解析贝叶斯参照的 Bernoulli 推断任务,在 3 个开源权重模型上比较 4 条文本路径和 2 条激活干预路径形成的信念状态。结果显示证据格式决定读出方式:结果序列主要影响续写,摘要和陈述比例主要影响显式概率估计。通过有效证据权重衡量发现,摘要的权重随其声称的观测数只呈弱缩放,提问方式可使推断权重相差数量级。单次激活干预虽然能复现自然记录的即时预测,但在后续新证据下效果大幅衰减。
Same Answer, Different Future: Matched Predictions Do Not Imply Matched Belief Updating in Language Models
Language models can produce similar probabilistic predictions after observing outcomes, reading summaries, receiving instructions, or undergoing activation interventions. Whether these routes induce equivalent belief states remains unclear: agreement in current predictions does not establish how a model will respond to subsequent evidence. We investigate this distinction in a Bernoulli inference task with an analytic Bayesian reference. Across three open-weight language models, we prepare states through four textual routes and two activation-level interventions, measure both next-outcome continuations and explicit probabilistic estimates, and append identical new evidence from each prepared state. We quantify the persistence of the initial belief through its effective evidence weight, the number of new observations it is worth. We find that the format of evidence decides which readouts express its content: outcome sequences primarily affect continuations, whereas summaries and stated rates primarily affect explicit estimates. Moreover, effective weights differ substantially across routes, with summaries receiving weights that scale weakly with their stated observation counts and elicitation questions changing inferred weights by orders of magnitude. In a one-shot steering setting, an intervention that reproduces a natural record's immediate prediction loses most of its effect under subsequent evidence. These results show that agreement in current readouts is insufficient to establish equivalence in belief updating: how a belief is acquired and how it is read jointly determine its subsequent dynamics.