MOMENT-DP 框架揭示动量会改变差分隐私相关噪声机制的评估结果
MOMENT-DP: Momentum Aware Evaluation of Correlated Noise for Differentially Private Training
做差分隐私训练的朋友看看这篇。作者发现只改动量这一项，隐私噪声机制的排名就会反转，以前只看曲率和学习率的评估方法可能不够。如果你在做私有逻辑回归或者 DP 微调，选噪声机制前最好把动量也算进去。
一篇 arXiv 论文提出 MOMENT-DP，这是一个感知优化器状态的评估框架，用于分析时间相关隐私噪声在基于动量的优化过程中如何传播。对于局部二次目标，该框架通过一个优化器传递算子来表达隐私带来的最终损失，同时刻画曲率、学习率调度、动量和时间噪声协方差四个要素，而此前的 NoiseCurve 曲率感知二次准则恰好是其零动量、恒定学习率的特例。
研究的关键发现是：在固定曲率、学习率调度、隐私校准和候选机制不变的情况下，仅改变动量这一项，就会导致相关噪声机制之间的排名发生真实反转。这一效应在逐样本裁剪的私有逻辑回归等非精确二次动态场景中依然存在，但在论文所研究的神经网络场景中尚未得到解决。
作者据此认为，优化器状态是评估时间相关隐私噪声时此前被欠建模、但直接影响决策的一个维度。对需要在小批量、多轮训练中做差分隐私的团队来说，这提示动量超参的选择可能直接影响隐私机制的有效性排序，评估时不应只看工作负载结构和曲率。
MOMENT-DP: Momentum Aware Evaluation of Correlated Noise for Differentially Private Training
Temporally correlated noise can substantially improve the utility of differentially private optimization, yet candidate mechanisms are commonly designed or evaluated using only partial information about the optimization process, such as workload structure, learning-rate schedules, or local curvature. We study whether persistent optimizer state provides additional information that can change the relative utility of privacy-valid correlated-noise mechanisms. We introduce MOMENT-DP, an optimizer-state-aware framework that models how temporally correlated privacy noise propagates through momentum-based optimization. For locally quadratic objectives, MOMENT-DP expresses privacy-induced final loss through an optimizer transfer operator that jointly captures curvature, learning-rate scheduling, momentum, and temporal noise covariance. NoiseCurve's curvature-aware quadratic criterion is recovered exactly as the zero-momentum, constant-learning-rate special case. Changing only momentum while holding curvature, schedule, privacy calibration, and candidate mechanisms fixed induces genuine mechanism-ranking reversals. The effect persists outside exact quadratic dynamics in per-example-clipped private logistic regression, while remaining unresolved in the neural regime we study. These results establish optimizer state as an under-modeled, decision-relevant axis in evaluating temporally correlated privacy noise.
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：差分隐私、MOMENT-DP、优化器、噪声机制，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「arXiv cs.LG」（@Sunny Gupta, Raja Viswanadha），经 AITOP 多维评分管线结构化提炼。
