Neural Expansion 框架从训练动力学解释深度网络泛化机制
Neural Expansion: A Unified Mechanism for How Deep Neural Networks Generalize
arXiv 新论文，用射线覆盖扩张这个几何视角解释网络为什么能泛化，还能覆盖对抗样本和误标注数据，做理论研究的可以看看。
一篇 arXiv 论文提出 Neural Expansion 几何框架，用方向选择性射线覆盖的扩张来刻画神经网络泛化。论文定义了 Generalization Intervals（GIs），量化训练样本周围模型预测保持稳定的方向特定区域。在多种架构和数据集上的实验显示，训练会使射线覆盖逐步扩张，尤其是沿低 Jacobian 奇异值的方向。该几何机制能解释大部分正确分类的测试输入、对抗样本以及误标注或分布外样本。
Neural Expansion: A Unified Mechanism for How Deep Neural Networks Generalize
Deep neural networks demonstrate remarkable generalization despite being highly over-parameterized, challenging traditional statistical learning theories. While several theoretical paradigms such as margin-based bounds, PAC-Bayesian analyses, and neural tangent kernels provide global explanations, they often lack mechanistic insight into how generalization emerges from training dynamics. We introduce Neural Expansion, a unified geometric framework that characterizes generalization through the expansion of directionally-selective ray coverage in the input space. Using Generalization Intervals (GIs), we quantify direction-specific regions around training samples where the model's predictions remain stable. Our extensive empirical studies across diverse architectures and datasets reveal that training leads to progressive expansion of ray coverage, particularly along directions with low Jacobian singular values. This mechanism helps explain a large fraction of correctly classified test inputs, adversarial examples, and even mislabeled or out-of-distribution samples through the geometry of ray coverage rather than instance-level memorization. By connecting input-space curvature to predictive behavior, our framework provides a mechanistic foundation that complements existing global theories of deep learning.
