论文

逻辑一致性给出无标签校准下界的锐利证明

Sharp and Stable Label-Free Calibration Certificates from Logical Coherence

精选理由

一篇很硬的理论论文,讲怎么在没有任何标注的情况下检验模型预测的校准程度,方法用逻辑一致性做抵押,还附了预注册实验。

arXiv 论文 2610.21567 研究 query-conditioned 概率预测器在缺失标注时的校准检验问题。作者用 minimax 定理导出有限支撑分布下校准残差的无标签下界,并证明锐利边际下界在答案上不连续,任何 Wasserstein 距离下连续的下界都无法保持常数因子。Lipschitz 抵押设定给出稳定下界,在线梯度上升在 i.i.d. 数据流上生成固定期限证书。预注册实验覆盖 4 个预测器、2 个数据集,7 个验证性证书全部为正。

原文 · arXiv cs.LG

Sharp and Stable Label-Free Calibration Certificates from Logical Coherence

Query-conditioned probabilistic predictors answer logically related questions one at a time, and labels to check their calibration are often missing. For each restriction on what a bettor's stakes may use, a minimax theorem identifies, for finitely supported laws of the answers, the sharp label-free lower bound on the corresponding calibration residual. On bundles of different events its mean, marginal and joint stages can separate strictly, and a bundle's bound cannot be attributed to one question without assumptions on the others. The sharp marginal bound is discontinuous in the answers, and no label-free lower bound continuous in the Wasserstein distance stays within a constant factor of it. Lipschitz stakes give a stable bound, sharp for a smooth calibration residual and, for every positive Lipschitz constant, positive exactly when the sharp marginal bound is. On i.i.d. streams, projected online gradient ascent at fixed nodes gives fixed-horizon certificates whose shortfall is bounded by grid, learning and statistical terms, and predictable betting gives anytime-valid ones. In a preregistered study of four predictors on two data sets, all seven confirmatory certificates were positive (the eighth pair did not pass the pilot). A certificate shows only that calibration is at least this bad; zero means "not refuted".