PresenceBench：评测大模型对用户状态的适应能力
PresenceBench: An Exploratory Evaluation of Large Language Model Adaptation to User State
四个主流模型谁更懂“你现在的状态”？这篇论文用五臂对照实测了，结果比想象中微妙，做对话产品的可以看看评测方法。
论文提出 PresenceBench，衡量模型能否根据用户的功能状态（准备度、心流、稳定性）调整回应。评测采用五臂因果设计，覆盖 Claude Sonnet 5、DeepSeek V4 Pro、Gemini 3.6 Flash、GPT-5.6 Terra 四个模型和 30 个场景。结果显示四个模型在提供状态信号后人类偏好点估计均高于 0.5，但八个 95% 区间中只有三个排除了偶然因素。特异性问题未解决：没有任何单模型区间排除偶然，设计还无法区分状态值本身与泛泛的调整指令。策略化传递状态降低了费用和输出 token，但 360 例中仍出现 13 例安全底线失败。
PresenceBench: An Exploratory Evaluation of Large Language Model Adaptation to User State
Presence-aware intelligence is the capacity of an AI system's intelligence layer to adapt appropriately to the functional state of the person it serves (here readiness, flow, and stability) while preserving task quality and safety. We introduce PresenceBench, an instrument for measuring it. PresenceBench evaluates how the intelligence layer uses a supplied state, not how that state is sensed, and breaks the capability into measurable properties. Two are tested here. Receptivity: does the model take up the state and give a better response than with no signal? Specificity: does any improvement depend on the correct state, rather than on extra structured context? A third, adaptation across states, is a required endpoint not measured in this release. A five-arm causal design answers both questions, against each model's own baseline and an equally structured mismatched-state control, scored by a correctness-gated human paired comparison. The exploratory evaluation covered four models across eighteen single-turn, six fixed three-turn and six simulated natural-conversation scenarios. Supplying a synthetic state, raw or translated into a response policy, gave human-preference point estimates above 0.5 for all four models (Claude Sonnet 5, DeepSeek V4 Pro, Gemini 3.6 Flash, GPT-5.6 Terra), with three of eight 95% intervals excluding chance. Specificity was not resolved: no per-model interval excluded chance, and the design cannot yet separate the state values from a generic instruction to adapt. Policy delivery reduced priced cost and output tokens against each model's baseline; these are resource-use changes, not efficiency gains, because correctness non-inferiority was not established. Safety-floor failures were rare (13 of 360 cases), with a non-significant excess under policy delivery. We report the instrument, the frozen results, and the controls a confirmatory study needs.
