Hidden not Deleted：神经网络如何抑制纠缠特征
Hidden not Deleted: How Networks Suppress Entangled Features
一篇讲特征擦除为什么失败的论文：线性投影擦不干净，遗忘的知识一个标量补丁就能恢复，还解释了 LLM unlearning 复现的机制。
论文研究了特征擦除方法在线性投影下的假设失效问题：当两个特征被压缩成共享同一子空间的对跖对时，SOTA 线性擦除会同时破坏两个特征。梯度下降训练的网络会根据初始化收敛到两种不同的电路级解，作者称之为 mirror 和 shadow 解。通过定向因果干预，论文证明两种解都会在擦除后留下可测量的特征表征痕迹，只需一个标量补丁即可恢复，无需进一步训练。这与 LLM unlearning 中观察到的遗忘知识复现现象吻合，为该失效模式提供了机制层面的解释。
Hidden not Deleted: How Networks Suppress Entangled Features
Concept erasure methods that operate via linear projection assume that features occupy separable subspaces. We show this assumption fails under dense superposition: when two features are forced into an antipodal pair sharing a single subspace, state-of-the-art linear erasure destroys both, not just the target. Networks trained with gradient descent instead solve this problem non-linearly, but not uniformly: they converge to one of two distinct circuit-level solutions depending on initialization, which we call mirror and shadow solutions. We map this bifurcation as a function of feature entanglement, show it reflects a stable attractor structure rather than an artifact of our setup, and use targeted causal interventions to demonstrate that both solutions leave a substantial, measurable trace of the erased feature's representation intact, recoverable through a single scalar patch rather than requiring any further training. This mirrors a failure mode recently observed empirically in LLM unlearning, where suppression rather than deletion allows forgotten knowledge to resurface; our results offer a mechanistic, causally-validated account of why that failure mode occurs.