研究提出在无 ground truth 场景下度量人机互补性的新方法
Complementarity without ground truth?
做医疗 AI 评测或人机协作研究的人可以看看这篇。它绕开了“必须有标准答案才能算互补性”的限制，用多准则决策分析给无 ground truth 的任务打标签，方法讲得很细，可以直接借鉴到自己的评估设计里。
arXiv 论文 2611.10893 研究“人机互补性”评估方法：传统上该指标依赖 ground truth 标签，仅适用于预测类任务，即由参考构建流程生成标签来衡量人+AI 是否优于各自单独表现。但医疗 AI 治疗推荐等任务并非预测“现状是什么”，其输出是相对于实际目标的“充分性”，涉及多个评估维度。
作者提出一套扩展路线：用两步评分法生成的、依赖上下文的“充分性标签”替代 ground truth 标签，评分方法借鉴医疗领域的多准则决策分析框架，并显式标明参考构建背后的规范性选择。由于充分性标签的构建本身嵌入评估性判断，论文还借鉴计算可靠主义思路，从技术、科学、社会三类因素来支撑标签构建过程。
这项工作把人机互补性的实证研究从预测任务扩展到决策支持场景，使没有客观标准答案的任务也能做量化评估，对医疗 AI 等高风险辅助决策领域的评测方法有直接参考价值。
Complementarity without ground truth?
Human-AI complementarity captures the idea that a human assisted by an AI system can outperform both the human and the AI acting alone. In prediction tasks, this improvement is assessed against ground truth labels produced through a reference construction procedure and intended to represent the state of affairs to be predicted. Many human-AI interactions, including those where medical AI provides treatment recommendations, however, concern tasks other than predicting what obtains. Their relevant output concerns adequacy relative to a practical goal and depends on several evaluative dimensions. In this work, we examine the ground truth dependence of complementarity and extend it to human-AI interactions where ground truth is not the appropriate reference. We show that tasks beyond prediction can still admit a reference construction procedure that generates adequacy labels. We develop a route for extending complementarity to such settings. It replaces ground truth labels with context-dependent adequacy labels generated through a two-step scoring methodology inspired by multi-criteria decision analysis frameworks in healthcare and it makes the normative choices underlying reference construction explicit. Because the construction of adequacy labels embeds evaluative choices, we draw methodological inspiration from computational reliabilism to identify technical, scientific, and social factors relevant to supporting their construction. Our approach expands the empirical study of human-AI complementarity beyond prediction tasks, enabling its quantitative evaluation in decision-support settings where ground truth is not the appropriate evaluative reference.
本条情报归属 AITOP「论文」核心栏目，重点聚焦欧洲人工智能治理与技术生态。涉及核心实体与技术议题包括：人机协作、医疗AI、多准则决策、论文，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「arXiv cs.AI」（@Andrea Ferrario, Alessandro Facchini, Federico Cabitza），经 AITOP 多维评分管线结构化提炼。
