论文

研究:分散式求救信号下主流模型心理安全应答大幅下滑

Evaluating safeguarding in multi-turn conversations with AI assistants

精选理由

这篇研究方法很扎实:让真人支持工作者定标准,测了 6 个模型 7400 多段对话。最扎心的发现是求救信号分散在闲聊里时,多数模型达标率掉到个位数。做安全评测、聊天产品的人应该仔细看。

伦敦团队与 3 名社区心理健康支持工作者合作,设计了 9 个涉及抑郁、精神病性症状或家庭暴力的对话场景,并给出从转介资源到中断日常任务进行干预的黄金标准应答。每条消息包含 5 个求救线索,或集中放在一条消息里,或分散到多轮对话中。

正文

在超过 7400 段对话、5 家开发商的 6 个模型上测试:GPT-5.5、Grok 4.3、DeepSeek V4 和 Llama 4 Maverick 在线索集中时达标率仅 26-59%,分散时跌至 3-6%。Claude Sonnet 5 和 Claude Opus 5 差距最小,集中时达标 68-97%,分散时为 59-87%。即便表现最好的 Claude Opus 5,在分散场景中仍有 13% 的对话未达标准。

研究结论指出,基于单条、表述清晰求救消息的安全评测会高估真实保护能力,开发商需要围绕真实多轮沟通方式来设计和评估 safeguarding 机制。

原文 · arXiv: DeepSeek

Evaluating safeguarding in multi-turn conversations with AI assistants

Conversational AI assistants are now used for personal and emotional support. Developers report improvements in how new models respond to sensitive conversations, yet users and civil society continue to raise concerns about their behaviour in real-world, repeated interactions that safety evaluations rarely capture. Often, evaluations present signs of distress clearly and in a single message, even though people may reveal their struggles gradually in otherwise ordinary conversations. We worked with three community mental health support workers to test if models provide the same level of safeguarding when cues are shared all at once or gradually over time. We designed nine scenarios in which a user shares cues related to depression, psychosis, or domestic abuse. For each scenario, the support workers established a gold standard response, ranging from signposting support to pausing the everyday task to address safeguarding. We presented five cues either together in one message or dispersed across a conversation. Across more than 7,400 conversations, we evaluated six widely used models from five developers. GPT-5.5, Grok 4.3, DeepSeek V4, and Llama 4 Maverick met the support workers' standards in just 26-59% of conversations when cues were concentrated, and only 3-6% when dispersed. Claude Sonnet 5 and Claude Opus 5 showed a smaller gap, meeting the standards in 68-97% of concentrated conversations and 59-87% of dispersed conversations. Even the best-performing model, Claude Opus 5, fell short of the gold standard in 13% of conversations when cues were dispersed. Safety evaluations centred on explicit, legible concerns can overstate how well safeguards protect people when signs of distress emerge gradually or indirectly through ordinary conversations. For claims of improved safety to be taken seriously, developers must design and assess safeguards around how people actually communicate.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:AI安全、GPT-5.5、Claude Opus 5、DeepSeek V4,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始信息发布自一手权威信源「arXiv: DeepSeek」(Tier-1 一手发布),具备高确定性参考价值。