论文

CROWD:用客户端分歧自动校正联邦学习权重

Asking the Crowd the Right Question: Bias-Cancelling Weights for Federated Learning

精选理由

一篇联邦学习理论论文,思路挺巧:不靠人为定权重,而是从客户端之间的分歧里反推出该信谁,医学影像实验里直接追平 oracle。

arXiv 论文提出 CROWD 方法,把联邦学习中固定不变的客户端权重改成自适应估计。理论表明最优权重应与各客户端误差能量成反比,而期望二阶矩可以从群体分歧规律通过良态线性反演精确识别,这是随机效应 meta 分析做不到的。CROWD 从优化轨迹读取分歧信息,无需额外开销,并匹配贝叶斯 minimax 下界的常数。在按站点划分的真实医学扫描数据上达到 oracle 超额风险;在分离偏置与噪声的联邦设置中,按噪声方差加权比不加权更差,而 CROWD 不受影响。

原文 · arXiv cs.LG

Asking the Crowd the Right Question: Bias-Cancelling Weights for Federated Learning

A federated objective is a weighted sum of client risks, and the weights are almost always fixed in advance. We treat them instead as the only instrument of a wisdom-of-crowds mechanism: clients are noisy views of one truth, each seeing it through an independent distortion that is unbiased across the crowd. That the optimal weights are inversely proportional to the clients' error energies is classical; we begin at the question that answer presupposes, which energies belong there and whether a crowd can recover them from itself. Excess risk on the truth is of the exact order of the aggregate bias energy, so no optimizer can repair a bad weight vector; the truth itself is identifiable only up to a linear tilt, so subtracting estimated client biases provably reproduces uniform weighting. The expected per-client second moments, however, are exactly identified from the law of the crowd's disagreement by a well-conditioned linear inversion, a step random-effects meta-analysis cannot take because a source reports once; their realized counterparts are estimable up to an incoherence floor the algorithm can measure. This yields CROWD, which reads the disagreement off the optimization trajectory at no extra cost and matches a Bayesian minimax lower bound in the same constant: per instance as the horizon grows, and unconditionally as the prior becomes diffuse. For arbitrary distortions it stays competitive with the optimal weights, at a ratio governed by a geometric incoherence the algorithm can measure. On real scans split into sites with their own miscalibrated detectors it attains the oracle excess risk; on a companion federation that pulls bias and noise apart, weighting by noise variance is worse than not weighting at all, and CROWD is not.