一次性分布式学习新论文：引入服务器数据改进加权平均的 MSE 界
One-shot Distributed Learning via Weighted Averaging with Server-side Information
一篇联邦学习理论论文：worker 数量超过一定规模时，普通加权平均会失效，但加上服务器自己的数据就能救回来，还给了具体的样本量条件。
arXiv 论文研究 m 个 worker（每个 n 条样本）加服务器 p 条数据的分布式学习场景。当 worker 数量 m = O(n^{2α-1}) 时，按样本量加权平均即可达到 pooled-data ERM 的 O(N^{-1}) MSE 阶；超过该规模则需要服务器样本量 p = Ω(nm·m/n^{2α-1})。论文还证明在某些规模下，仅用服务器数据的估计（MSE 为 O(p^{-1})）反而优于加权平均。实验部分在 CIFAR-10 上比较了线性与非线性分类器下的不同聚合策略。
One-shot Distributed Learning via Weighted Averaging with Server-side Information
Distributed learning has drawn considerable attention due to its capability of learning from distributed data without aggregating raw observations. However, frequent exchange of model updates can become the main bottleneck of communication. Therefore, one-shot approaches where each worker fits a local model and sends it to the server to aggregate emerge. It is known that averaging can achieve the so-called pooled-data mean squared error (MSE) order of a hypothetical node that has access to data from the server and all the workers, and performs empirical risk minimization (ERM) on it. However, when the number of workers increases too fast compared to the local sample size, nonzero local estimation bias prevents averaging from achieving this order. In this paper, we study whether independent server data can overcome this limitation. We consider $m$ workers with $n$ observations each and a server with $p$ observations, all i.i.d., giving $N=mn+p$ observations in total. Under certain regularity conditions, a pooled-data ERM achieves MSE of order $\mathcal O(N^{-1})$. We analyze deterministic nonnegative averaging of convex-loss M-estimators, with equal worker weights and the server's own estimator included. Under our assumptions, including nonzero local bias of order $n^{-α}$, $α\geq1$, sample-size weighted averaging achieves the pooled-data MSE order whenever $m=\mathcal O(n^{2α-1})$. Above this worker scale, it does so if and only if $p=Ω(nm\cdot m/n^{2α-1})$. Hence, server-only estimation, whose MSE is of order $\mathcal{O}(p^{-1})$, can achieve pooled-data order while sample-size weighting fails. Analytical examples and synthetic experiments illustrate when averaging matches the pooled-data MSE order, complemented by CIFAR-10 experiments comparing aggregation strategies with linear and nonlinear classifiers.