论文

MOMASPLAT:面向移动操作的新视角示教增强框架

MOMASPLAT: Dynamic Novel-View Demonstration Augmentation for Mobile Manipulation

精选理由

机器人研究必看:用 3D Gaussian Splatting 把示教视频重渲染成新视角,成功率翻倍还省采集成本。

移动操作机器人因导航误差和定位漂移,很少能从示教采集时的底盘位姿执行操作,视觉运动策略对此高度敏感。MOMASPLAT 用 3D Gaussian Splatting 结合前馈 3D 基础模型,从标准数据采集的多视角观测中重建静态场景和被操作物体,并基于运动学进行动作重定向,生成包含动态物体交互的完整合成示教。该方法在 3 个场景、5 项真实世界移动操作任务(含刚体和关节物体)上评估,将最强基线的平均成功率提高一倍以上,且在大位移底盘位姿下仍保持非零成功率。

原文 · arXiv cs.AI

MOMASPLAT: Dynamic Novel-View Demonstration Augmentation for Mobile Manipulation

Visuomotor policies trained through imitation learning are highly sensitive to changes in camera viewpoint. This is particularly problematic for mobile manipulation (MoMa), where robots rarely execute manipulation from the exact same base pose used during demonstration collection due to navigation uncertainty, localization drift, and environmental constraints. Recent approaches use novel-view synthesis (NVS) to augment demonstrations with unseen viewpoints, but they assume static scenes and therefore fail to re-render the manipulated object during the post-contact interaction phase, leaving the most critical parts of the demonstration unaugmented. We present MOMASPLAT, a novel-view demonstration augmentation framework for mobile manipulation that synthesizes full manipulation trajectories from unseen robot base poses, including dynamic object interactions. Our key insight is that standard MoMa data collection naturally provides the multiview observations needed to reconstruct both the static scene and the manipulated object in 3D. MOMASPLAT combines 3D Gaussian Splatting, feed-forward 3D foundation models, and kinematics-based action retargeting to separately reconstruct the scene and manipulated object, render them from novel viewpoints, and generate geometrically consistent synthetic demonstrations paired with transformed actions. We evaluate MOMASPLAT on five real-world mobile manipulation tasks involving both rigid and articulated objects across three scenes, and validate the fidelity of its synthesized demonstrations against ground truth in simulation. MOMASPLAT substantially improves robustness to unseen base poses, more than doubling the average success rate of the strongest baseline and maintaining non-trivial success at large base displacements where all baselines fail, while reducing the need for largescale real-world data collection.