COP 方法让 LLM 持续遗忘不伤模型能力
Preserving Utility in Continual LLM Unlearning via Orthogonal Projections
模型要删隐私数据又怕越删越笨？这篇论文的 COP 方法用正交投影解决顺序遗忘互相干扰的问题，TOFU 和 RWKU 上都验证了效果。
论文提出 Continual Orthogonal Projection（COP），一种面向 LLM 的持续遗忘（unlearning）梯度投影方案。它把每次删除更新限制在与历史遗忘集激活子空间正交的补空间中，保证新删除不干扰之前的删除。COP 每层只存一个低秩基，不需要过去的梯度或遗忘集，可直接接入任何通过可微损失梯度步骤更新模型的 unlearning 方法。在 TOFU 和 RWKU 两个基准上，COP 在顺序删除请求下保持了模型效用，遗忘效果也优于持续遗忘专属基线。
Preserving Utility in Continual LLM Unlearning via Orthogonal Projections
Unlearning concerns the task of removing the influence of a sensitive portion of the training data, such as protected, copyrighted or private text, from a model without retraining it from scratch. While unlearning requests happen continuously during a model's lifetime, the vast majority of LLM unlearning methods are ill-suited for iterated removal, yielding steep decay in forgetting and utility upon sequential unlearning. This decay stems from interference between successive updates, as each new deletion can undo earlier ones or push the model further along directions already degraded by them. To address this, we introduce Continual Orthogonal Projection (COP), a gradient-projection scheme that restricts each update to the orthogonal complement of activation subspaces cached from prior forget sets. This approach provably guarantees first-order non-interference with earlier deletions. COP stores only a low-rank basis per layer, requiring neither past gradients nor previous forget sets, and plugs into unlearning methods that update the model through gradient steps on a differentiable loss without modifying its objective. Experiments on standard LLM unlearning benchmarks (TOFU and RWKU) show that COP preserves utility across sequential requests without weakening forgetting, preventing the collapse of standard objectives and forgetting more effectively than continual-specific baselines.