ReShap：面向高维多动作环境的可解释强化学习框架
ReShap: Interpretable Reinforcement Learning for High-Dimensional Multi-Action Settings
把 SHAP 归因和策略蒸馏结合，让玩 Atari 游戏的深度 RL 智能体也能被看懂，决策树代理比线性回归更忠实。做可解释 AI、安全 RL 或想在机器人等高风险场景部署 RL 的研究者值得复现。
论文提出 ReShap，一个基于 Shapley 归因的策略蒸馏框架，将 SHAP 可解释性扩展到高维状态空间和多动作场景，突破了此前 SILVER 框架仅支持低维状态、二值动作和单一线性代理模型的限制。
技术路径上，ReShap 从原始图像观测中提取紧凑特征表示，用 SHAP 做特征归因以刻画每个特征对策略决策的贡献，再把深度 RL 策略蒸馏到可解释代理模型上，训练数据覆盖策略访问分布上的全部 (state, policy-action) 对。代理模型实例化了决策树、线性回归和逻辑回归三类模型族。
作者在 Atari 和 Box2D 环境中用三种深度 RL 算法评估，并开展了人类受试者实验检验策略的清晰度与可信度。结果显示 ReShap 保持有竞争力的任务回报，决策树在忠实度上普遍优于线性回归和逻辑回归，线性回归在多数场景中表现最弱。
ReShap: Interpretable Reinforcement Learning for High-Dimensional Multi-Action Settings
Deep Reinforcement Learning (RL) plays a crucial role in tackling complex tasks, yet its reliance on deep neural networks makes the learned policies opaque. To enhance the trustworthiness of RL policies, existing work (e.g., the SILVER framework [Li et al., 2025]) has focused on Shapley-based interpretability, but is typically restricted to low-dimensional state spaces, binary-action settings, and a single class of linear surrogate. We propose ReShap, a policy-distillation framework that extends Shapley-based interpretability to high-dimensional, multi-action environments. ReShap extracts compact feature representations from raw image observations, performs SHAP-based feature attribution to characterize how each feature contributes to the policy's decisions, and distills the deep RL policy into an interpretable surrogate trained on the full set of (state, policy-action) pairs along the policy's visited distribution. We instantiate the surrogate with three interpretable model families (decision trees, linear regression, and logistic regression) and compare their fidelity and task return. We evaluate ReShap on Atari and Box2D environments using three deep RL algorithms and conduct a human-subject study to assess the clarity and trustworthiness of the resulting policies. Results show that ReShap preserves competitive task performance, that decision trees generally outperform linear and logistic regression, and that linear regression is weakest in most settings.
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：ReShap、SHAP、强化学习、可解释AI，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「arXiv cs.LG」（@Yiyu Qian, Pengfei Guo, Liyuan Zhao），经 AITOP 多维评分管线结构化提炼。