论文提出 Generative Montage:真实信息片段也能让 AI 智能体相信谣言
Language-model agents' beliefs can be manipulated via truthful evidence in open channel
LLM 智能体读社交媒体时,即使看到的都是真帖也会拼出谣言,14 个模型里 12 个中招率超 65%。论文给的 Grounded Monitor 能提 5.6 到 29 个百分点,做多智能体系统的人值得看看防御思路。
arXiv 论文提出 Generative Montage 攻击方法,借鉴电影蒙太奇理论与叙事认知理论,协调智能体把真实的信息片段拼成误导性叙事,由此产生的失败模式被命名为 narrative overfitting。实验覆盖六个 Twitter 事件中的 1,090 个谣言目标,14 个模型中有 12 个的谣言接受率超过 65%,且经多数投票或重新审阅同样证据后,接受率仍高于 50%。
论文通过片段干预定位影响信念的具体帖子,并用合成对照证明相似的回应模式可由有效证据触发,从而区分信念影响与证据支持。基于这一区分,作者提出神经符号运行时监控器 Grounded Monitor,通过检查直接证据支持来纠正智能体判断,在原始与匿名化设置下将决策准确率提升 5.6 到 29.0 个百分点。
这项研究揭示了谣言通过社交渠道影响智能体信念的认知路径。对构建多智能体系统、让智能体读取社交媒体信息的开发者来说,监控推断出的关系在成为结论前的状态,是防止错误传导到下游决策的具体抓手。
Language-model agents' beliefs can be manipulated via truthful evidence in open channel
Language-model agents' beliefs can be manipulated through open channels such as social media, even when the information they receive is truthful. When agents fill gaps between fragments with self-induced relationships, they may accept those relationships as established facts and pass the resulting rumours to others. Drawing on cognitive theory of narrative understanding and film theory of montage, we introduce Generative Montage to operationalize this failure, termed narrative overfitting, by coordinating agents to arrange truthful fragments into misleading narratives. Across 1,090 rumour targets from six Twitter events, agents often accepted rumours overconfidently, with overall acceptance exceeding 65 percent in twelve of fourteen models. Acceptance remained above 50 percent after majority voting or review of the same evidence, showing how errors can reach downstream decisions. To explain this susceptibility, fragment interventions identify posts that influence belief within complete narratives. Synthetic controls then show that similar response patterns arise from valid evidence, separating belief influence from evidential support. This distinction motivates Grounded Monitor, a proof-of-concept neuro-symbolic runtime monitor that checks direct evidential support to guide agents' judgements, improving decision accuracy by 5.6 to 29.0 percentage points across original and anonymized settings. These findings expose a cognitive route to rumour amplification and support monitoring inferred relationships before agent-generated conclusions guide public decisions.
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:智能体、AI安全、Grounded Monitor、narrative overfitting,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Jinwei Hu, Qisong He, Xinmiao Huang, Youcheng Sun, Yi Dong, Xiaowei Huang),经 AITOP 多维评分管线结构化提炼。