RESCUE：用反事实评估方法检验机器人运行时监控告警是否值得干预
RESCUE: Is This Alarm Worth Acting On? Counterfactual Evaluation of VLA Runtime Monitors
这篇论文问了个体面问题：报警器响了到底该不该管？用 12 万多个反事实分支做实验，结论挺反直觉。
论文提出 RESCUE 框架，通过在模拟器中回滚状态生成反事实分支，评估 VLA 运行时监控的告警能否带来有效干预。实验覆盖 OpenVLA-OFT 和 π0.5 两个模型，在 LIBERO-Spatial 和 LIBERO-Plus 上共运行 128,340 个反事实分支。结果显示 τ=0.5 时，66 个 OpenVLA-OFT 失败案例中仅 14 个可在续跑策略下恢复，改用脚本化恢复后升至 34 个。在 stop-and-reset 干预模型下，即使是理想检测器，误报率超过 2.0–3.3% 就会让不必要的重置抵消提前停止的收益。
RESCUE: Is This Alarm Worth Acting On? Counterfactual Evaluation of VLA Runtime Monitors
Runtime failure monitors for robot policies are typically evaluated by how accurately they distinguish failed from successful executions. Yet an accurate alarm may arrive too late for recovery, while an unnecessary intervention may disrupt an execution that would otherwise succeed. We introduce RESCUE, a counterfactual framework for evaluating whether a monitor's alarms enable useful interventions. By restoring simulator and policy states at execution checkpoints, RESCUE compares alternative actions, identifies the latest checkpoint satisfying a specified recovery criterion, and evaluates whether acting on alarms reduces expected task-completion cost. We evaluate RESCUE with OpenVLA-OFT and $π_{0.5}$ over LIBERO-Spatial and LIBERO-Plus using 128,340 counterfactual branches. Recoverability depends on the state--intervention pair rather than on robot state alone: at $τ=0.5$, 14 of 66 OpenVLA-OFT/LIBERO-Plus failures contain a recoverable checkpoint under continuation, compared with 34 of 66 under continuation or scripted recovery, while recovery reduces recoverability in other conditions. On the primary strong-policy conditions, favourable detection scores do not establish lower intervention cost. Under the evaluated stop-and-reset model, even an idealised detector can afford only a 2.0--3.3% false-alarm rate before unnecessary resets outweigh the savings from stopping failures early. These results show that monitor value depends jointly on the intervention available and the workload in which it is deployed.
