RACER:从少量样本估计陌生专家能力的人机路由框架
RACER: Role-Aligned Competence Estimation for Human-AI Routing
AI 什么时候该把判断交给人?RACER 从少量样本就能估计陌生专家的准确率,还在放射科胸片数据上验证过。
这篇 arXiv 论文提出 RACER(Role-Aligned Competence Estimation for Routing),解决 learning to defer 问题:预测系统何时自主行动、何时交给人类专家。针对 L2D-Pop 可能学到依赖绝对类别坐标的路由捷径、IFD 的能力估计在类内恒定这两个缺陷,RACER 用角色相对方式估计陌生专家在查询上答对的 posterior-predictive 概率,再与模型后验结合。论文证明了类别重标记不变性,并给出 Bayes 对齐的 defer 代理损失与 plug-in 后悔界。在 CIFAR-100 合成实验的未见专家划分上取得最强总体表现,在 PathMNIST 病理上下文扩展研究中随上下文增多而受益。在放射科医生参与的胸片基准 VinDr-CXR 和 CheXpert 上,RACER 系列在预算扫描 defer 中持平或最优。
RACER: Role-Aligned Competence Estimation for Human-AI Routing
Learning to defer asks a predictive system when to act autonomously and when to defer to a human expert. Population-adaptive deferral extends this problem to unseen experts using a small context set of expert behavior. Neural context encoders such as L2D-Pop can be query-dependent, but may learn routing shortcuts tied to absolute class coordinates. Identity-Free Deferral (IFD) removes such shortcuts through role-indexed classwise competence profiles, but its estimates are constant within each class and cannot capture instance-level expert specialization. We propose RACER---Role-Aligned Competence Estimation for Routing---a role-relative framework for estimating an unseen expert's competence from context. RACER estimates the posterior-predictive probability that the expert is correct on a query under each candidate class role, then combines these estimates with the model posterior to obtain the Bayes-relevant expert-correctness probability. Nonparametric and neural kernel-pooling estimators use candidate-role relations, shared aggregation, and symmetric summaries, excluding absolute class-identity channels. We prove coherent class-relabelling invariance, derive a Bayes-aligned deferral surrogate, and give a plug-in regret bound relating routing regret to classifier and competence-estimation error. On controlled synthetic benchmarks, including a PathMNIST histopathology context-scaling study with simulated experts, RACER benefits from additional context under hidden subtype dependence and gives the strongest aggregate performance on a separately sampled unseen-expert split in the CIFAR-100 synthetic experiments. On the radiologist and human--AI chest-radiography benchmarks (VinDr-CXR and CheXpert), the RACER family is competitive or best in budget-swept deferral, with calibration results varying across metrics and datasets.