论文

论文:无标签预算下利用免费协变量加速风险监控

The First-Order Value of Free Covariates in Budgeted Risk Monitoring

精选理由

模型上线后想及时发现风险升高,但标注很贵。这篇论文算出了理论极限,还展示利用输入数据免费触发能省 1/b 轮延迟。

arXiv 论文 2611.00576 研究在标签预算受限时如何监控已部署模型的风险上升。论文证明,仅依赖自身标签历史的规则在标签率 b=c/log A 下,最坏情况(Lorden)延迟至少为 (1/h(r)+1/c)log A。当有害变化同时引起输入分布偏移时,允许协变量触发标注的规则在高斯模型上可将一阶延迟降到 log A/h(r),即免费协变量至少节省 1/b 轮等待。混合规则为输入不变的风险变化加入标签历史回退,合成实验和两个 ACS 任务验证了预测结果。

原文 · arXiv cs.LG

The First-Order Value of Free Covariates in Budgeted Risk Monitoring

A deployed model sees its inputs for free, but checking whether its risk has risen requires costly labels. We study how to spend a label budget so that a risk increase is detected quickly while false alarms stay controlled, including under input shifts that keep the risk below a safety threshold $τ$. Let $A$ be the false-alarm control scale and $h(r)=\mathrm{kl}(r,τ)$ the Bernoulli KL divergence between the changed risk $r$ and $τ$. At label rate $b=c/\log A$, we prove that every rule that chooses requests from its own label history has worst-case (Lorden) delay at least $(1/h(r)+1/c)\log A$ to first order: besides collecting evidence, it must wait for its next label. The bound needs only a label budget on one finite stretch of safe source data and false-alarm control over an initial window; a bound on the average time to false alarm does not suffice. When a harmful change also shifts the inputs, a rule that lets the inputs trigger labeling attains delay $\log A/h(r)$ to first order on a fixed Gaussian model, so free covariates save at least $1/b$ rounds. A hybrid rule adds a label-history fallback for risk changes that leave the inputs unchanged. Synthetic experiments and two ACS tasks, with strengthened label-only baselines, show the predicted pattern. The covariate trigger has a cost: because its budget is calibrated on the source, a benign input shift can make it request nearly every label.