RACER：用风险敏感目标与Critic一致性正则改进对抗强化学习
Robust Adversarial Reinforcement Learning with Risk Sensitivity and Critic Consistency Regularization
做强化学习的可以看看：RACER 用自适应扰动和 critic 一致性正则，解决了对抗训练里对手太狠把智能体带偏的问题，连续控制实验效果确实比老基线稳。
arXiv 论文提出 RACER 框架，针对 RARL（Robust Adversarial Reinforcement Learning）中优化不稳定的问题给出两项改进。一是引入状态依赖的对抗目标，自适应调节扰动强度，避免过强扰动把智能体推向无效失败状态。二是加入 critic 一致性正则，缩小双 Q 值估计器之间的分歧，稳定价值学习。在连续控制基准上的实验显示，RACER 在性能、鲁棒性和训练稳定性上均优于现有鲁棒 RL 基线。
Robust Adversarial Reinforcement Learning with Risk Sensitivity and Critic Consistency Regularization
Reinforcement learning (RL) achieves strong performance in sequential decision-making but remains brittle under dynamic uncertainty and distributional shifts. Robust Adversarial Reinforcement Learning (RARL) improves robustness via worst-case perturbations, but existing approaches frequently suffer from unstable optimization and degraded value estimation. In particular, overly aggressive adversaries can drive the agent toward uninformative failure states, while adversarial perturbations amplify disagreement between double critics and introduce biased value targets. We propose a unified framework, RACER (Risk-sensitive robust Adversarial critic ConsistEncy-regularized Reinforcement learning), that revisits adversarial RL from a risk-sensitive perspective. First, we introduce a state-dependent adversarial objective that adaptively regulates perturbation strength, suppressing harmful disturbances while preserving informative exploration. Second, we propose critic consistency regularization to reduce disagreement between Q-value estimators and stabilize learning. Comprehensive experiments on challenging continuous control benchmarks demonstrate that RACER consistently improves performance, robustness, and training stability over strong robust RL baselines.
