arXiv 论文提出单体化鲁棒学习:单数据集训练多个假设降低联合失效概率
Monoculture Robust Learning
搞理论或关心模型同质化风险的人可以看这篇。它把 2022 年的可复制性理论和 2021 年的算法单体化问题接上了:同一个数据集跑 k 次可复制学习器,联合出错率就能指数级下降,代价是样本量要多 Ω(log(1/γ)) 倍。比独立采数据集省钱,思路对做集成和风险分析的人有直接参考价值。
一篇 arXiv 论文(编号 2611.24881)在 PAC 学习框架下提出"单体化鲁棒性"概念,针对 Kleinberg 和 Raghavan(2021)指出的算法单体化问题:共享训练数据的模型往往在同一批输入上同时出错。论文的核心问题是:仅使用一个数据集的学习器能否输出 k 个各自准确、且联合失效概率接近用 k 个独立数据集训练的假设。
技术上,作者建立了可复制性(Impagliazzo 等,2022)与单体化鲁棒性之间的联系:若基础学习器满足 ρ-可复制,则在同一样本上独立运行 k 次,个体单体化误差会随 k 指数级下降,下降速度由基础学习器的单次误差和 ρ 控制。
论文同时给出下界:要把个体联合误差压到 γ,样本量需要乘以 t = Ω(log(1/γ)) 的因子,说明该方法存在固有的样本代价。
Monoculture Robust Learning
We introduce a notion of monoculture robustness in Probably Approximately Correct (PAC) learning, as a way to mitigate the problem of algorithmic monoculture (Kleinberg and Raghavan, 2021): the tendency of models sharing the same components (e.g., training data) to fail on the same inputs. Specifically, we ask whether a learner with access to a single dataset can output $k$ individually accurate hypotheses whose joint failure probability is comparable to training on $k$ independent datasets. We say that such a learning procedure achieves monoculture robustness, and we quantify this by bounding the conjunctive error that all $k$ hypotheses misclassify points in an individual sense. We design methods for achieving this type of guarantee by drawing a surprising connection between replicability (Impagliazzo et al., 2022) and monoculture robustness. Specifically, we show that if the base learner is $ρ$-replicable, then running it independently $k$ times on the same sample yields individual monoculture error that shrinks exponentially with $k$, controlled by the single-run error of the base learner at the point and $ρ$. We complement these positive results with a lower bound: for individual monoculture robustness, we prove a worst-case lower bound showing that, for learners, driving the individual conjunctive error down to $γ$ can require a sample-size multiplier of $t = Ω(\log(1/γ))$.
本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。涉及核心实体与技术议题包括:PAC学习、可复制性、算法单体化、机器学习理论,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「arXiv cs.LG」(@Lee Cohen, Jon Kleinberg, Omer Reingold),经 AITOP 多维评分管线结构化提炼。