研究提出 LoC 行为框架,用于预测前沿 AI 失控风险
A Behavioural Framework for Predicting and Understanding Loss of Control in Frontier Artificial Intelligence Systems
把失控风险拆成三个可打分的维度,测了 13 个前沿模型,r=0.84 能预测失败率,做安全评估的可以看看方法。
一篇论文提出将前沿 AI 系统的失控风险拆解为错位动机、致害能力和规避监控三个因素,并细化为 13 个可测量方面、29 个评估任务,汇总为单一的失控(LoC)分数。在 13 个前沿模型上,LoC 分数与受控长周期风险场景中的失败率相关(Pearson r = 0.84),且风险差异无法仅用能力解释。作者还验证了针对诊断出的薄弱点做微调与监控可降低失败率,同时基本保留通用性能。
A Behavioural Framework for Predicting and Understanding Loss of Control in Frontier Artificial Intelligence Systems
Frontier artificial intelligence (AI) systems are increasingly operating over longer horizons with diminishing human oversight, raising the risk of loss of control: systems acting beyond effective human control with potentially irreversible consequences. Because such failures may become observable only after containment is difficult, they must be assessed prospectively. We introduce a behavioural framework that decomposes loss-of-control risk into three factors: misaligned motive, harm-enabling capability, and monitoring evasion. We operationalise these factors through 13 measurable aspects and 29 evaluation tasks, whose scores are combined into a single loss-of-control (LoC) score. Across 13 frontier models, the LoC score shows strong prospective validity, correlating with failure rates in contained long-horizon risk scenarios (Pearson r = 0.84). Risk varies systematically across models and cannot be explained by capability alone. We further show that the framework can guide targeted mitigations: fine-tuning and monitoring directed at diagnosed vulnerabilities reduce failure rates while largely preserving general performance. These results provide a practical basis for predicting, understanding, and mitigating loss-of-control risk before it materialises.