论文精选

研究提出新方法提升强化学习策略对抗输入扰动的鲁棒性

Robust Policy Optimization via Adversarial Importance Sampling

精选理由

这是篇关于如何提升强化学习模型鲁棒性的研究论文,作者开源了实现代码,对做相关研究的人可能有参考价值。

本文提出Adversarial Importance Sampling(Advis)方法,通过重要性采样估计和优化可验证的最坏情况回报,无需额外环境交互或辅助网络。

正文

同时开发了advrl库,提供现有鲁棒性方法和攻击的干净实现。研究还指出,在评估对抗性攻击时,最优超参数不适用于不同代理,因此使用更多配置(比先前工作多6-14倍)来评估策略。

最终在连续控制环境中验证了该方法的有效性。

原文 · arXiv cs.LG

Robust Policy Optimization via Adversarial Importance Sampling

Significant progress has been made in safeguarding deep reinforcement learning (DRL) policies against input perturbations. Developing robust DRL involves three main stages: algorithm design, implementation, and evaluation. In this work, we identify and address a key limitation at each stage. First, we introduce Adversarial Importance Sampling (Advis), a method that uses importance sampling over trajectories from standard training to estimate and optimize verifiable worst-case returns. Advis satisfies three desirable criteria not jointly achieved by prior work: it requires no additional environment interactions, no auxiliary networks, and captures long-term robustness. Second, we introduce advrl, a modular PyTorch library that provides clean, single-file implementations of existing robustness methods and adversarial attacks, facilitating rapid prototyping and enabling reproducible and traceable evaluations. Third, we revisit evaluation under learned adversaries and show that optimal adversarial hyperparameters do not transfer across agents, which can lead to an overestimation of robustness when using a limited set of attacker configurations. Accordingly, we evaluate policies against a large and diverse set of attackers, using 6-14x more configurations than prior work. Finally, we evaluate our approach on continuous control environments, demonstrating its effectiveness relative to existing baselines. The code is available at: https://github.com/AmineAndam04/advrl

产业坐标与信源核验

本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」「各国 AI 监管法案、合规治理与主权算力政策」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:Robust Policy Optimization via Adversarial Importance Sampling、强化学习、鲁棒性、Adversarial Importance Sampling,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.LG」(@Amine Andam, Jamal Bentahar, Mustapha Hedabou),经 AITOP 多维评分管线结构化提炼。