基于表示质量的紧致泛化边界：k-NN 证书摆脱参数空间复杂度
Escaping Parameter Space: Tight Generalization Bounds via Representation Quality
有人用 k-NN 分类器给神经网络算出了很紧的泛化证书，CIFAR-10 上只差 8 个点，ImageNet 上也能算出具体数字，做理论的可以看看。
论文提出用最后一层隐藏层表示加无参数 k-NN 分类器来度量神经网络泛化边界，不再依赖权重范数、KL 散度等参数空间复杂度指标。从零训练的 ResNet-18 在 CIFAR-10 上与测试误差仅差 8.0 个百分点，并在 CIFAR-100 上给出首个非空洞证书。使用冻结的 DINOv2 表示时，证书在 CIFAR-10 达到 1.4%，在 1,000 类 ImageNet 上达到 23.1%。该边界可分解为局部类几何、解码器-网络一致性和训练稳定性三个可独立测量的量，实验证明破坏局部类几何会消除泛化能力。
Escaping Parameter Space: Tight Generalization Bounds via Representation Quality
We derive a tight generalization bound for deterministic neural We derive a tight generalization bound for deterministic neural networks based on their last hidden layer representations. By evaluating representation quality with a parameter-free $k$-nearest-neighbor ($k$-NN) classifier, our certificate avoids traditional parameter-space complexity measures, KL divergences to a posterior, weight norms, and compression. The bound is tight: trained from scratch, it reaches an 8.0 percentage point gap to test error on CIFAR-10 (ResNet-18) and yields the first non-vacuous certificates we are aware of on CIFAR-100; with frozen DINOv2 representations, it certifies at 1.4\% on CIFAR-10 and 23.1\% on 1,000-class ImageNet. It decomposes additively into three empirical, independently measurable quantities: local class geometry ($k$-NN decoder error), decoder-network agreement, and training stability. This allows the bound to act as a diagnostic tool: we demonstrate that actively destroying local class geometry eliminates generalization while leaving dataset memorization intact, identifying class geometry as a causally relevant mechanism for generalization. Finally, we show that learned representations are significantly less sensitive to weight noise than linear readouts, suggesting why parameter-space certificates often struggle to remain tight.