研究提出统一框架分析梯度遗忘方法的输出空间行为
Loss is Not Behavior: A Unified Output-Space Analysis of Gradient-Based Machine Unlearning
一篇关于机器遗忘的理论论文，告诉你为什么 loss 降了不等于真忘了，还给了能提前算的诊断工具，做模型隐私相关的可以看看。
arXiv 论文《Loss is Not Behavior》指出，基于梯度的机器遗忘方法只优化 forget 与 retain 损失，但损失相近不代表输出行为接近重训练模型。作者在线性回归中推导出输出空间差距的闭式表达，并通过广义 Gauss-Newton 与局部线性近似扩展到非线性模型。框架用几何、重加权、投影三个维度统一现有方法，并提出 forget self-leverage 与 retain cross-leverage 两个杠杆矩阵，理论证明它们决定 forget 残差对输出的影响。论文还给出可预计算的杠杆诊断，用于审计 forget 集并判断哪种梯度遗忘方法更可能成功。
Loss is Not Behavior: A Unified Output-Space Analysis of Gradient-Based Machine Unlearning
Gradient-based machine unlearning methods typically ``unlearn'' by optimizing forget and retain losses, but similar losses do not necessarily imply similar output-space behavior. Prior work has shown that high forget loss alone does not certify that unlearning has produced retraining-like output behavior. We therefore ask: when a gradient-based unlearner differs from the model that would have been learned on only the retain data, what structure does the resulting output-space gap have? We derive closed-form expressions for this gap in linear regression, with an extension to nonlinear models via a generalized Gauss-Newton and local linearity approximations. Our analysis puts various gradient-based methods under a unified framework parametrized by geometry, reweighting, and projection. Two leverage matrices emerge that capture the forget-retain entanglement under the retrained model geometry: forget self-leverage and retain cross-leverage. We show both theoretically and empirically that these two leverage matrices drive the impact of the forget residual on the forget and retain outputs, and existing methods account for this geometry only partially. Our framework yields precomputable leverage diagnostics for auditing forget sets to identify what type of gradient-based unlearning is likely to be successful.