强化学习让通用机器人策略实现多任务在线自我改进
Reinforcement Learning Enables Scaling Generalist Robot Policy Improvement
机器人版通用模型怎么在部署后越用越强？这篇论文给出了 RL 改进的具体配方，讲清了哪些设计选择决定泛化效果。
arXiv 论文（编号 2611.06905）研究了通用机器人策略能否在部署阶段同时改进多个任务。作者设计了一套结构化评测，系统比较了价值估计、策略提取和资源分配等算法设计选择。实验表明，用基于价值的 RL 配合合适的训练任务分布，通用策略不仅能在线提升已见任务，还能泛化到在线训练未覆盖的任务。作者称这是机器人基础模型在线通用自我改进的首批证据之一。
Reinforcement Learning Enables Scaling Generalist Robot Policy Improvement
The introduction of generalist robot policies has enabled a single, large model to learn to perform a variety of tasks from a vast pre-training corpus. While these models often exhibit powerful zero-shot generalization capabilities, they typically require an adaptation phase to achieve sufficient performance in deployment. Iterated self-improvement via reinforcement learning (RL) has demonstrated promise in enabling this adaptation, yet such approaches have been largely explored in single-task settings - where the goal is to simply improve on a narrow deployment task - which lies in stark contrast to the multi-task capabilities of the base policy. In this work we ask whether generalist robot policies are capable of multi-task, generalist improvement in deployment - targeting many diverse tasks simultaneously - and what algorithmic recipe enables this. To answer this question, we first develop a set of structured evaluations to test generalist improvement capabilities, and systematically evaluate algorithmic and practical design choices regarding value estimation, policy extraction and resource allocation. We show that not only can generalist policies be improved online across diverse tasks, but that the right recipe can enable generalization and increase performance on tasks not considered during online training. We characterize when this generalization occurs and show that both the training task distribution and the use of value-based RL are critical. To the best of our knowledge, this is among the first evidence for online generalist improvement for robot foundation models.