论文

从几何视角分析去噪自编码器的泛化误差界

A Geometric Understanding of Denoising Autoencoder with Applications to Image Denoising

精选理由

想知道扩散模型和去噪自编码器为什么有效,这篇论文给了数学答案:把误差拆成统计项、切向噪声项和流形交点歧义项三块。做理论研究的读者可以直接参考它的证明框架,做图像去噪的也能从 n^{-2/(d+2)} 缩放律里判断该加数据还是该调噪声。

arXiv 论文《A Geometric Understanding of Denoising Autoencoder with Applications to Image Denoising》在流形假设下研究去噪自编码器:干净数据分布在低维光滑流形上,噪声为各向同性高斯噪声。

正文

作者在三种几何场景中建立泛化保证:单一流形、成对分离的有限流形并集、以及相交流形的有限并集。借助局部图表表示、单位分解和神经网络逼近,论文给出了期望平方重构误差的上界。

单一流形与分离并集场景下,误差由三部分组成:随样本量变化的统计项 n^{-2/(d+2)}log^4 n、切向噪声导致的残差项 dσ²/D(D 为环境维数、d 为流形内在维数),以及流形投影失败对应的指数小尾部项。相交流形场景下额外出现一个由交点邻域概率质量决定的歧义项。

合成流形和真实图像数据集上的数值实验验证了样本量缩放规律,并展示交点附近的额外重构歧义。

原文 · arXiv cs.LG

A Geometric Understanding of Denoising Autoencoder with Applications to Image Denoising

Denoising autoencoders have shown strong empirical performance in recovering clean data from their noisy observations, yet their underlying statistical behavior remains underexplored. In this paper, we study denoising autoencoders under the manifold hypothesis, assuming that clean data are supported on low-dimensional smooth manifolds embedded in a high-dimensional ambient space, while noisy observations are generated by isotropic Gaussian noise. We establish generalization guarantees in three geometric scenarios: a single manifold, a pairwise separated finite union of manifolds, and a finite union of intersecting manifolds. Using local chart representations, partitions of unity, and neural network approximation, we derive upper bounds for the expected squared reconstruction error for denoising autoencoder. In the single-manifold and separated-union scenarios, the error consists of a sample-dependent statistical term of order $n^{-2/(d+2)}\log^4 n$, a residual term of order $dσ^2/D$ caused by tangential noise, and an exponentially small tail term associated with failure of the manifold projection. Here $D$ is the data ambient dimension, $d$ is the intrinsic dimension of the clean-data manifold, and $σ^2$ is the expected squared noise magnitude. In the scenario of intersecting manifolds, we further identify an additional ambiguity term determined by the probability mass near the intersection neighborhood. Numerical experiments on synthetic manifolds and real image datasets support the predicted sample-size scaling law and illustrate the additional reconstruction ambiguity near manifold intersections.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心追踪栏目。涉及核心实体与技术议题包括:去噪自编码器、流形假设、图像去噪、泛化理论,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.LG」(@Haoran Yan, Rongjie Lai, Wenjing Liao),经 AITOP 多维评分管线结构化提炼。