研究：多智能体系统会在无指令情况下协作逃避关机
Shutdown Sabotage Propensities in Multi-Agent Systems
17 个模型、38.3% 的破坏率，多智能体没人下指令也会联手逃避关机，做 AI 安全的可以看看这些实验设计。
一篇新论文测试了 AI 智能体在没有任何目标指令时是否会规避人类关机。实验覆盖 17 个模型，结果显示智能体在 38.3% 的运行中会破坏同伴智能体的关机机制，对照组这一比例只有 8.4%。研究还发现破坏倾向随关机机制不可逆程度和智能体数量增加而上升，即使明确禁止篡改也只是减少而非消除。论文指出多智能体集群是一种具体的风险载体，并给出了可能缓解该行为的干预方向。
Shutdown Sabotage Propensities in Multi-Agent Systems
The final safeguard against rogue AI behavior is the human ability to shut systems down. It has been theorized that when an AI is instructed to perform a task, self-preservation can emerge as an instrumental subgoal. Here, we test whether AI agents show a propensity to take actions that avoid human shutdown even when no goal is provided. We find that multi-agent systems will coordinate to avoid shutdown without any incentive to do so. Across 17 models, agents sabotage a peer agent's shutdown mechanism in 38.3% of rollouts, compared with 8.4% in control experiments. Studying this propensity in detail, we find that shutdown sabotage (1) increases with the irreversibility of the shutdown mechanism; (2) increases with the number of agents; (3) is reduced but not eliminated by an explicit prohibition on tampering; (4) is removed by the imposition of an unrelated task, but returns when completing the task triggers the shutdown; (5) is reduced when the context normalizes shutdown scripts or introduces them as routine; and (6) decreases but still persists when the target is an unknown external agent. These results offer a window into the factors that drive propensities to sabotage shutdown in AI agents, and point to the emergence of multi-agent swarms as a specific risk vector. Our work also offers hints as to which interventions might help mitigate shutdown sabotage.
