论文

DISCERN:无需保留集的增量式机器遗忘方法

DISCERN: Distributional Synthetic Replay for Retain-Free Incremental Machine Unlearning

精选理由

删数据不用重新训练模型,这论文给出了不用保留原始数据的增量遗忘方案,transformer 上删 20 类还很稳。

机器遗忘要求从已训练模型中删除指定类别数据的影响,但实际场景中删除请求会陆续到来。DISCERN 不访问保留集,而是为每个类别拟合参数分布(高斯、拉普拉斯、Student's t、von Mises–Fisher、Fisher–Bingham),用合成嵌入替代真实回放,并通过统计检验选择分布。在 MNIST、CIFAR-10、CIFAR-100 上对比七个无需保留集的基线,覆盖卷积和 transformer 两类骨干。增量删除 20 个类别后,ViT-B/16 上 DISCERN 调和均值保持 0.797,而同时删除场景下最强的基线从 0.918 降至 0.048。消融实验表明按类别选择分布能显著减少已遗忘类别的复现。

原文 · arXiv cs.LG

DISCERN: Distributional Synthetic Replay for Retain-Free Incremental Machine Unlearning

Machine unlearning aims to remove the influence of specific training data from a trained model without retraining from scratch. In real deployments, however, deletion requests arrive over time, calling for incremental class unlearning (ICU): classes are removed sequentially, previously forgotten knowledge must remain forgotten, and the remaining classes must be preserved. We present DIstributional SynthetiC rEplay for Retain-free iNcremental unlearning (DISCERN), an intervention-based class unlearning method that replaces access to the retain set with synthetic embeddings sampled from per-class parametric distributions. Unlike prototype-based replay, which assumes an isotropic Gaussian around each class centroid, DISCERN fits Euclidean (Gaussian, Laplace, Student's $t$) and directional (von Mises--Fisher, Fisher--Bingham) families to each class and selects among them through statistical tests and, when none of them accepts, cross-validated log-likelihood. This design is motivated by the architecture-dependent latent geometry: transformer embeddings lie approximately on a hypersphere and concentrate around class directions, unlike those of convolutional networks. We evaluate DISCERN against seven retain-free baselines on MNIST, CIFAR-10, and CIFAR-100 with two convolutional and two transformer backbones, under single-class, multi-class, and incremental unlearning. DISCERN is competitive when classes are removed at once and is the most stable method in ICU on transformers: on ViT-B/16, after 20 sequential deletions it retains a harmonic mean of $0.797$, whereas the strongest method in the simultaneous setting drops from $0.918$ to $0.048$. A paired ablation shows that per-class distribution selection substantially reduces the resurfacing of previously forgotten classes, acting as a stabilizer of accumulated forgetting rather than a uniform improvement.