DECODE:诊断 LLM 自动可解释性流程中的失效问题
DECODE: Diagnosing Interpretability Failures in Automated Interpretation Process
现在不少可解释性管线是 LLM 自己解释自己,没人盯很容易翻车。DECODE 用解释间共识加主动播种把人拉回审查环节,还在 16384 个 SAE 特征上跑通了案例。做机制可解释性、SAE 特征分析的团队可以直接参考它的三阶段诊断流程。
可解释性研究正越来越多地让 LLM 同时生成和评估对模型内部表征的解释,形成缺乏人工监督的自指自动化流程。由于缺少 ground truth,研究者很难验证解释质量,也难以判断失效源自特征提取工具还是流程中的 LLM 本身。
DECODE 是一套可视化分析系统,把诊断工作流拆解为三个渐进阶段,隔离从结构缺陷到解释层面错位的各类失效来源。方法上借鉴质性研究中的编码员间共识,用独立生成解释之间的共识度及评估指标作为可靠性代理;再通过三阶段主动播种协议,优先推荐高优先级实例供人工审查。
案例研究覆盖 16,384 个 Sparse Autoencoder 特征,验证了该系统在大规模场景下识别并归因可解释性失效的能力。
DECODE: Diagnosing Interpretability Failures in Automated Interpretation Process
Tools for interpreting the internal representations of Large Language Models (LLMs) increasingly rely on LLMs themselves to both generate and evaluate explanations, creating a self-referential automated interpretation process with minimal human oversight. This automation raises significant reliability concerns, which are exacerbated by the absence of ground truth. In practice, it is difficult to validate explanations or to attribute interpretability failures to their source -- whether they originate from the tool's extracted representations or from the LLM used in the process. To address these challenges, we present DECODE, a visual analytics system that reintroduces human oversight into the automated interpretation process. Our approach decomposes the diagnostic workflow into three progressive stages, isolating failure sources ranging from structural defects to explanation-level misalignments. Drawing inspiration from inter-coder consensus in qualitative analysis, we utilize consensus among independently generated explanations and their evaluation metrics as a proxy for reliability. To further scale human inspection, we employ a three-phase active seeding protocol that strategically recommends high-priority instances for manual review. Finally, we demonstrate the effectiveness of our approach through a case study on 16,384 Sparse Autoencoder features, showing that it can efficiently identify and attribute interpretability failures at scale.
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:Sparse Autoencoder、LLM 可解释性、可视化分析、主动学习,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Dohyun Kim, Jaeung Lee, Donghee Hong, Jaemin Jo),经 AITOP 多维评分管线结构化提炼。