论文提出 AdaSGDA：用正交化更新与自适应步长改进 minimax 优化稳定性
Orthogonal Updates for the Win: Towards Accelerated Adaptive Minimax Optimization
做 GAN、对抗训练这类 minimax 优化的人应该看看这篇。它用正交化更新加自适应步长，解决了 SGDA 系方法对步长调参极敏感的问题，且 AdaSGDA-VR 的 O(T^-1/3) 收敛率比常见基线更快。
一篇 arXiv 论文针对单环 minimax 优化方法在原始-对偶强耦合下更新震荡、对步长极其敏感的问题，提出正交更新框架，沿正交化方向施加更新以降低更新方向的各向异性。
正交化会改变优化几何结构，使原始和对偶两侧步长的协调变得更困难。论文据此提出 AdaSGDA，以累计梯度范数驱动的自适应步长规则配合正交更新，在非凸-强凹 minimax 优化中实现接近免调参的求解。理论方面，AdaSGDA 在无需依赖问题调参的前提下达到 O(T^-1/4) 收敛率；其加入方差归约机制的变体 AdaSGDA-VR 进一步达到更快的 O(T^-1/3) 收敛率。
实验显示两种方法相对现有 minimax 优化方法具有竞争力的性能。minimax 优化常见于 GAN 训练、对抗学习与强化学习等场景，对步长敏感这一痛点长期存在，该工作为相关方向提供了免调参的替代方案。
Orthogonal Updates for the Win: Towards Accelerated Adaptive Minimax Optimization
Single-loop minimax methods are appealing for modern machine learning, but standard stochastic gradient updates can be unstable and oscillatory under tightly coupled primal-dual dynamics, making performance notoriously sensitive to stepsizes. To fundamentally improve stability, we develop an orthogonal update framework tailored to minimax optimization, which applies updates along orthogonalized directions to reduce update anisotropy. However, orthogonalization reshapes the optimization geometry and makes coordinating primal-dual stepsizes even more delicate. To overcome this challenge, we develop AdaSGDA, which equips orthogonal updates with an adaptive stepsize rule driven by accumulated gradient norms. This mechanism automatically balances coupled primal-dual progress under orthogonalization, yielding an effectively parameter-free method for nonconvex-strongly-concave minimax optimization. We prove that AdaSGDA avoids problem-dependent tuning while attaining the state-of-the-art $\mathcal{O}\left(T^{-1/4}\right)$ convergence rate. We further propose AdaSGDA-VR, which incorporates a carefully designed variance reduction scheme and achieves the faster $\mathcal{O}\left(T^{-1/3}\right)$ rate. Extensive experiments demonstrate that our methods deliver competitive performance compared to existing minimax optimization methods.
本条情报归属 AITOP「论文」核心追踪栏目。涉及核心实体与技术议题包括：minimax优化、AdaSGDA、优化算法、方差归约，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「arXiv cs.LG」（@Zhiwei Zhai, Xinyu Wang, Wenjing Yan, Lei Ding, Ying-Jun Angela Zhang），经 AITOP 多维评分管线结构化提炼。