Anthropic 红队警告：小型模型或数月内具备自主复制能力
Meta: "we finally have a frontier model!" Google: "Gemini can output 1M tokens now 🤯" Anthropic:...
Anthropic 红队说小型模型几个月内可能自己复制自己，还不用数据中心，这篇把复制链拆开讲了，安全向的必看。
Anthropic Frontier Red Team 发布博客称，小型 AI 模型可能在数月内实现脱离数据中心的自主复制。报告指出，让模型成为强大编程智能体的能力同样可泛化到网络攻击任务，包括自主复制，2024 年编程智能体的情况正在 AI 蠕虫身上重现。团队尚未观察到完整的"流氓复制体"，但智能体已能完成复制链中的各个步骤，并把改动传递给自身的后续副本。报告预测可能出现寄生于不安全算力上的慢速"数字黏菌"、高频入侵数据中心的方法，或骗取算力额度的模型，建议各方加固系统并准备应对这类复制体长期存在的局面。
Meta: "we finally have a frontier model!" Google: "Gemini can output 1M tokens now 🤯" Anthropic:...
Meta: "we finally have a frontier model!" Google: "Gemini can output 1M tokens now 🤯" Anthropic: "we're reiterating that our AI is going to kill you" Andrew Curran @AndrewCurran_ Anthropic's Frontier Red Team believes small AI models may now be months away from being capable of self replicating without access to a datacenter. Quoting from the blog: 'The capabilities that make AI models generally powerful coding agents also generalize to cyber tasks, including autonomous replication. Like coding agents in 2024, we’re seeing signs of life for AI worms. We don’t yet have a full rogue replicator, but agents are becoming increasingly capable of completing the individual steps in the replication chain (and passing on their adaptations to future copies of themselves). At the same time, increasing amounts of inference compute are now available both inside and outside of datacenters (see Appendix D). It’s uncertain exactly what form autonomous AI replication will take. It could be a slow, distributed “digital slime mold,” living on relatively insecure compute. It could also be a powerful cyber model that breaches datacenters at a high pace, or a model that scams people for compute credits. As a prediction that goes beyond our experiments, we expect a future awash with agents running on all types of hardware, including some that aren’t under human control. While it’s true that IoT devices have been vulnerable for a long time, lightweight agents may be much more willing than human attackers to put in the effort to compromise large fractions of the world’s IoT compute. The rise of slow replicators would also be a bellwether for more aggressive, faster ones that can breach more hardened machines or be effective API key thieves. We should aim to harden our systems to make it difficult for such replicators to spread, but also prepare to deal with a world where they are here to stay.' 🔗 View Quoted Tweet 💬 13 🔄 3 ❤️ 38 👀 1716 📊 12 ⚡
