UniIntervene++：自适应干预智能体提升机器人在线强化学习效率
UniIntervene++: An Adaptive Intervention Agent for Efficient Real-World Reinforcement Learning
机器人在线 RL 训练时最麻烦的就是人工干预太多，这篇把它降到 0.77%，还开源了代码，做真机 RL 的可以看看。
论文提出 UniIntervene++，一个自适应干预智能体，将演化中的 RL 策略、轨迹修正和任务结构化 CodePolicy 统一建模为半马尔可夫决策过程中的 Options。它通过周期性无辅助执行探测 RL 策略的能力，动态决定何时干预、如何干预以及何时交还控制权。在 5 个真实世界机械臂操作任务上，UniIntervene++ 平均成功率达 89.67%，超过所有基线至少 6 个百分点，人工干预占比降至 0.77%，相比最佳基线相对减少至少 94.6%。代码已在 GitHub 开源。
UniIntervene++: An Adaptive Intervention Agent for Efficient Real-World Reinforcement Learning
Online reinforcement learning (RL) enables robot policies to improve through physical interaction, but the assistance they require changes as their competence evolves. Existing intervention strategies based on offline estimates or fixed decision rules can therefore become mismatched to the current policy. To address this, we propose UniIntervene++, an adaptive intervention agent that learns to allocate control between autonomous execution and heterogeneous assisted behaviors during online RL. Specifically, UniIntervene++ first formulates the evolving RL policy, trajectory correction, and a task-structured CodePolicy as Options in a unified semi-Markov decision process and learns their relative values online. Building on this, competence-adaptive intervention periodically probes the RL policy through unassisted execution, keeping control allocation responsive to its evolving capability. Finally, coupled experience learning allows assisted behaviors to improve the RL policy, whose evolving outcomes in turn reshape future intervention decisions. In this way, UniIntervene++ jointly determines when to intervene, how to intervene, and when to return control as the RL policy improves. Across five real-world manipulation tasks, UniIntervene++ achieves an average success rate of 89.67%, outperforming all baselines by at least 6 percentage points, while reducing human intervention to 0.77%, a relative reduction of at least 94.6% from the best baseline. Code is available in our \href{https://github.com/dannyyudong/An-Adaptive-Intervention-Agent-for-Efficient-Real-World-Reinforcement-Learning}{GitHub repository}.
