评估解释忠实度的各项标准为何互相矛盾：一个统一分析框架
Navigating Disagreement Among Criteria for Evaluating Explanation Faithfulness
做可解释性审计的朋友看这篇：同一解释在不同忠实度指标下结论打架，论文给了一套框架帮你定位分歧出在哪，还有 adverse action notices 的实际案例拆解。
这篇 arXiv 论文针对同一个解释在不同忠实度评估标准下结论不一致的问题展开分析。作者提出一个 faithfulness functional，把各评估标准的分歧定位在解释对模型做了什么声明、哪些比较与该声明相关、以及如何聚合比较结果这三个维度上，并在该框架内重构现有评估标准。论文以 adverse action notices（不利决定通知）为案例，重构了行业提出的评估程序，指出其中哪些选择由场景正当化、哪些仍未确定。结论是忠实度是一个定义不完全的关系，操作化方案的辩护需要诉诸解释性、制度性与规范性考量。
Navigating Disagreement Among Criteria for Evaluating Explanation Faithfulness
Faithfulness requires that algorithmic explanations accurately reflect the model's behavior. Yet existing operationalizations of faithfulness can disagree about whether the same explanation is faithful, leaving developers and auditors to choose among conflicting evaluations. In this paper, we investigate disagreement among evaluation criteria and offer a practical path forward for evaluating explanation faithfulness. We first identify how operationalizations of faithfulness differ. We introduce a faithfulness functional that locates their disagreements in what an explanation claims about the model, which comparisons bear on that claim, and how agreement across those comparisons should be aggregated, and we reconstruct existing evaluation criteria within it. We argue these reconstructions support understanding faithfulness as a common though incompletely specified relation between the model and explanation, so the differences reflect choices left open by the general requirement. Justifying an operationalization then comes down to justifying these choices, which, on our view, draws on explanatory, institutional, and normative considerations. We support this argument with a case study of adverse action notices, reconstructing the evaluation implicit in an industry-proposed procedure and identifying which of its choices the setting justifies and which remain open. Our work gives auditors and practitioners a way to navigate disagreeing evaluations of faithfulness. It also explains why the literature should expect multiple operationalizations, while showing how concrete settings can narrow the choice among them. More broadly, our analysis shows how a shared requirement of faithfulness can support contextually justified evaluations despite disagreement among existing metrics.
