论文

研究显示:仅靠删除数据的机器遗忘可能必须记住更多训练信息

Why Forget-Only Unlearning Needs Memorization

精选理由

这篇 arXiv 论文证明了一个反直觉结论:想让模型“只给模型就能删数据”,可能得让它记住几乎全部训练集,对做遗忘的研究者很有参考价值。

arXiv 论文 2610.10519 研究了 forget-only unlearning:删除算法只拿到训练好的模型和待遗忘样本,没有保留数据。论文证明不同数据集可能训练出相同模型,但删除同样样本后需要完全不同的输出,并据此推导出遗忘精度匹配重训练的下界。论文还给出记忆下界:对简单 threshold learner,支持任意删除请求所需记忆的信息量可以高达整个数据集,而普通训练只保留一个边界点。结论是普通训练中丢弃的信息可能日后删除时需要,因此为 forget-only unlearning 设计的模型需要保留比标准训练更多的信息。

原文 · arXiv cs.LG

Why Forget-Only Unlearning Needs Memorization

Machine unlearning asks for a deletion algorithm whose output is close to retraining from scratch without the selected forget examples. In this work, we study forget-only unlearning, where the deletion algorithm receives only the trained model and the examples to forget, with no retained data or extra training information. We ask whether forget-only unlearning is always possible. We first show that this depends on the learning method: different datasets can produce the same trained model but require very different outputs after the same examples are removed. Using this observation, we derive lower bounds on how accurately unlearning can match retraining and instantiate them for several standard learning algorithms. We then ask what must be true when forget-only unlearning succeeds. To this end, we derive lower bounds on what an algorithm must memorize about the training data to handle arbitrary deletion requests. For simple threshold learners, the required information can be as large as the entire dataset, even though ordinary training keeps only one boundary point. Overall, our results show that information discarded during ordinary learning may be needed later for deletion, so models designed for forget-only unlearning may need to retain more information than standard training does.