LIBERO-RIG：评估视觉-语言-动作模型硬件鲁棒性的基准
LIBERO-RIG: A Hardware-Setup Robustness Benchmark for Vision-Language-Action Models
机器人挪个位置、换个相机就不行？这基准造了 210 种硬件状况考 9 个 VLA 模型，最高难度成功率掉到 0.509。
LIBERO-RIG 是一个只扰动机器人硬件与设置、保持 LIBERO 任务内容不变的鲁棒性基准，覆盖相机安装与光学、相机-机械臂标定、执行器增益、控制频率、本体感知噪声、底座位姿、桌面高度、夹爪垫等 11 个扰动轴。每个轴设 5 个严重度等级，在四个 LIBERO 套件上共产生 210 种条件。团队以每条件 500 回合评估了 9 个近期视觉-语言-动作模型，整体成功率从 0.932 降至中等严重度的 0.767，最高严重度仅剩 0.509。各扰动轴和动作接口带来的性能下降差异明显，而在视觉扰动上做训练无法缓解不改变相机画面的硬件故障。基准连同逐回合扰动参数清单已一并开源。
LIBERO-RIG: A Hardware-Setup Robustness Benchmark for Vision-Language-Action Models
Robot policies are expected to generalize across deployment environments, including variations in the robot itself: cameras are bumped and remounted, calibrations drift, controllers run at different rates, and end-effector pads wear down. We present LIBERO-RIG, a benchmark that perturbs only the robot's hardware and setup while holding every LIBERO task fixed. The benchmark spans eleven perturbation axes, including camera mounting and optics, camera-to-arm calibration, actuation gain, control rate, proprioceptive noise, base pose, table height, and gripper pads. Each axis has five severity levels, yielding 210 conditions across the four LIBERO suites. We evaluate nine recent vision-language-action models using 500 episodes per condition and substantiate each finding with rigorous statistical tests. Pooled success falls from 0.932 to 0.767 at the middle severity and 0.509 at the highest severity. Performance degradation varies substantially across perturbation axes and action interfaces, and training on visual perturbations does not mitigate faults that leave camera images unchanged. We release the benchmark together with manifests specifying the fixed perturbation parameters for each episode.
