论文

论文提出对抗性多重性:模型披露可能被操纵绕过审计

Adversarial Multiplicity: Cherry-Picked Models and the Limits of Disclosure

精选理由

一篇挺有意思的论文:有人能在不动准确率的情况下悄悄把模型预测掰向自己要的方向,连审计都能过,监管和用模型打官司的人该看看。

arXiv 论文 2611.07378 研究预测多重性带来的被操纵风险:在几乎不损失预测性能的前提下,足够灵活的模型可被硬操纵(直接扰动参数)逆转任意个体预测,并通过黑盒审计。软操纵(超参调优、特征工程等常规选择)影响较小,但能通过要求完全访问代码和数据的白盒审计。论文举例称,模型对气候变化与野火关系的效应估计可被硬编辑和软搜索完全反转;Reconcile 算法虽能解决大部分模型间分歧,但当对立模型被对手对抗性选择时,结果更常偏向操纵方。作者建议采用预注册、训练证明、敏感性分析、多指标评估和版本化公开模型等缓解措施。

原文 · arXiv cs.LG

Adversarial Multiplicity: Cherry-Picked Models and the Limits of Disclosure

Predictive multiplicity -- where similarly accurate models disagree on individual predictions -- can be leveraged to optimize secondary metrics (e.g., fairness) without loss in predictive performance. However, exploiting multiplicity opens the opportunity for motivated parties (or coding agents gaming their objectives) to steer predictions toward their own ends, such as in litigation or consumer disputes with automated decision-making tools. In these adversarial multiplicity settings, we distinguish soft manipulation, where parties optimize over innocuous model choices (e.g., hyperparameter tuning, feature engineering), from hard manipulation, where parties directly perturb model parameters. We find a sufficiently flexible model can be manipulated to reverse any individual prediction with minimal change in predictive performance while passing black-box audits, making hard manipulation viable in most regulatory settings. Soft manipulation has smaller effects but can pass white-box audits requiring full access to code and data. Even aggregate estimates can be manipulated: a model's estimated effect of climate change on wildfires can be fully reversed by hard edits and soft search. We find the Reconcile algorithm typically resolves most disagreements between models, but more often in favor of the manipulator when the opposing model is chosen adversarially compared to at random. We map which manipulations can plausibly pass existing regulations and disclosure policies, and recommend mitigations such as pre-registration, proof of model training, sensitivity analyses, multi-metric evaluations, and committing to versioned public models.