Cleanly-DINO:用少量演示为机器人清洁任务学习视觉任务空间
Cleanly-DINO: Efficiently Learning Visual Task Spaces for Robotics
机器人清洁任务没法用物体位姿来定义,这篇论文给出新思路:冻结 ViT 特征加少量演示就能拟合任务空间和进展度量,还能当强化学习的奖励信号,最后真机擦拭验证。做机器人操作学习的研究者值得跑一遍方法细节。
UC Berkeley 等机构在 arXiv 发布论文 Cleanly-DINO(arXiv:2611.20137),提出从少量演示中学习视觉任务空间的方法,面向清洁这类以视觉状态变化定义的任务。
与刚性物体操作依赖物体位姿不同,清洁任务由视觉状态变化定义,任务表示的选择一直不明确。方法上,Cleanly-DINO 以任务相关区域的掩码和冻结的 ViT 特征作为输入,拟合一个投影来隔离任务空间,并在其上拟合进展度量(progression metric)。
团队在多样化的清洁示例上训练模型,评估其定位未完成任务区域和跨视觉域迁移的能力。在机器人学习场景中,模型被用作强化学习设置的观察编码器和奖励信号。
最终,团队将拟合好的模型部署到真实世界的机器人擦拭(wiping)流程中,验证其引导任务执行的效果。该工作展示了基于基础模型视觉特征构建任务表示的可行性,对少样本机器人操作学习有参考价值。
Cleanly-DINO: Efficiently Learning Visual Task Spaces for Robotics
Learning manipulation tasks from few demonstrations requires abstractions that capture the changes relevant to task completion, such as object poses in rigid-body manipulation. However, tasks such as cleaning are defined by changes in visual state rather than object pose, making the choice of task representation less clear. In this work, we propose Cleanly-DINO to learn visual task spaces from few demonstrations. Given a mask of a task-relevant region and frozen ViT features, Cleanly-DINO fits a projection to isolate a task space and fit a progression metric to it. We train the model on diverse cleaning examples and evaluate its ability to localize incomplete task regions and transfer across visual domains. We also study its utility for robot learning by using it as an observation encoder and reward signal in a reinforcement learning setting. Finally, we deploy our fitted model within a real-world robotic wiping pipeline to evaluate its use in guiding task execution.
本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:Cleanly-DINO、ViT、机器人学习、少样本学习,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「arXiv cs.LG」(@Adrian Röfer, Melika Shekarriz, José Arce y de la Borbolla, Iman Nematollahi, Abhinav Valada),经 AITOP 多维评分管线结构化提炼。