论文

研究刻画 Lipschitz 约束判别器在生成建模中的渐近行为

Lipschitz-Constrained Discriminators in Generative Modeling: Asymptotic Behavior and Support Mismatch

精选理由

搞 GAN 或对抗训练的人值得看看这篇:它把判别器里 Lipschitz 约束参数 L 从小到大的行为变化讲清楚了,尤其是训练初期模型分布和数据分布没有重叠时会发生什么。比起只看 WGAN 那套经验做法,这篇给出了理论层面的解释。

arXiv 论文 2611.18394 研究了 L-Lipschitz 正则化 KL 散度对应的最优判别器的渐近行为,该散度在重标度 Wasserstein-1 度量与 KL 散度之间插值。参数 L 充当 Lipschitz 约束和正则化参数:较小的 L 使判别器更依赖几何结构,L 增大时几何正则化逐步放松,已有结果表明该散度在 L 趋于无穷时收敛到 KL 散度。

正文

论文在三种情形下刻画了判别器的渐近行为:一是 Q 关于 P 绝对连续且 Radon-Nikodym 导数有上下界;二是 Q、P 为满足绝对连续条件的离散分布;三是两者为不满足绝对连续的离散分布。

作者指出该结果可直接指导含判别器的训练策略,如对抗式生成模型。实践中数据分布常用离散经验测度表示,训练早期模型分布与数据分布可能几乎没有重叠,此时支撑集不匹配的分析尤为关键。

原文 · arXiv cs.LG

Lipschitz-Constrained Discriminators in Generative Modeling: Asymptotic Behavior and Support Mismatch

We study the asymptotic behavior of optimal discriminators associated to $L-$Lipschitz regularized KL-divergences of two probability distributions $Q$ and $P$ that interpolate between the rescaled Wasserstein$-1$ metric by some parameter $L>0$ and the KL divergence. The parameter $L$ plays the role of a Lipschitz constraint and is viewed as a regularization parameter: small values of $L$ enforce a more geometry-aware discriminator, while as $L$ increases the geometric regularization is progressively relaxed. It is a known result [3] that such divergences converge to the KL divergence as $L\to\infty$. In this paper, we characterize the asymptotic behavior of the optimal discriminators as $L\to\infty$ in different regimes: Firstly, under $Q\ll P$ and the Radon-Nikodym derivative $\frac{dQ}{dP}$ is bounded from above and bounded away from zero. Secondly, when $Q$ and $P$ are discrete distributions satisfying $Q\ll P$, and finally when $Q$ and $P$ are non absolutely continuous discrete distributions. This result motivates training strategies which involve discriminators, such as adversarial generative models. In practice, the data distribution is often represented by a discrete empirical measure, while, especially at the early stages of the training, the model distribution may have little or no overlap with the empirical data distribution.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:生成模型、GAN、Wasserstein、KL散度,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.LG」(@Eduardo Pinto, Panagiota Birmpa),经 AITOP 多维评分管线结构化提炼。