Pixel-UMI：从人类演示数据学习环境感知的全身操控策略
Pixel-UMI: Learning Environment-Aware Whole-Body Manipulation from Sensorized Human Demonstrations
一篇机器人操控方向的新论文，用人类演示数据训练机器人全身动作，五项任务平均完成度 87%，做具身智能的可以看看方法细节。
Pixel-UMI 是一套从传感化人类演示数据学习机器人全身操控策略的流水线。其核心组件 Environment-Aware Retargeting（EAR）将重建的场景几何与头部、夹爪轨迹结合，生成避免碰撞的关节和底盘目标。针对人机形态差异导致的视角不匹配，Retargeting-Error-Aware Masking（REAM）在训练中利用重定向误差降低部署时对失配视角的依赖，并通过 DAgger 在策略上收集人类纠正。在关节物体操作、可变形物体处理、双臂协调等五项任务上平均任务进度达 87.0%。
Pixel-UMI: Learning Environment-Aware Whole-Body Manipulation from Sensorized Human Demonstrations
Sensorized human demonstrations offer scalable data for mobile manipulation, but embodiment differences complicate policy learning. We present Pixel-UMI, a pipeline for learning environment-aware whole-body policies from human data. We first introduce Environment-Aware Retargeting (EAR), which combines reconstructed scene geometry with demonstrated head and gripper trajectories to determine robot posture and generate collision-aware joint and base targets. Delay-calibrated action assignment converts these trajectories into actions. However, kinematic differences can leave residual pose errors, creating viewpoint mismatches between human demonstrations and robot observations. We therefore introduce Retargeting-Error-Aware Masking (REAM), which uses these errors during training to reduce reliance on mismatched views at deployment. Finally, because estimated actions cannot fully capture contact-rich interactions, we collect on-policy human corrections through DAgger to improve performance. We evaluate Pixel-UMI on five challenging tasks spanning articulated-object manipulation, deformable-object handling, constrained-space reaching, whole-body collision avoidance, and contact-rich bimanual coordination, achieving 87.0% average task progress. Video results are available at https://pixel-robot.com/pixel-umi