论文

JEPA 世界模型学会回答"如果当初":反事实推理无需反事实数据

Counterfactual World Models: JEPAs That Reason About "What If"

精选理由

这篇论文挺有意思:它从数学上证明普通世界模型天生答不了"换个动作会怎样",然后给出一个零额外数据的修正公式。实验里反事实 R² 直接从四成拉到九成,用的还是 DINOv2 这类现成冻结编码器。做机器人或强化学习的开发者值得认真读,反思机制可以直接套到已有模型上。

一篇 arXiv 论文证明:仅在事实轨迹上训练的世界模型,无论数据量或容量多大,都无法回答"在同样扰动下换一个动作会怎样"这类反事实问题。论文利用"自然无跳跃"(natura non facit saltus)这一物理性质——当动作与扰动作为独立力作用时,二阶交互项可忽略——推导出反事实结果的一阶近似:观测结果加上世界模型在替代动作下的预测差值。该方法不需要反事实数据、额外参数或重新训练,并给出 O(h²) 的反事实误差上界与匹配的极小极大下界。

正文

实验在像素层面用 JEPA 世界模型结合冻结的 DINOv2 和 MAE 编码器验证:四个渲染模拟器上,反事实 R² 从 38–46% 提升到 86–96%;第五个水下航行器模拟器上,用 16 倍事实数据达到 90–94%。DINOv3、SigLIP 2、CLIP、EVA-02 四个冻结编码器在同一组模拟器上也取得 81–95%。在事后规划(hindsight planning)任务中,该方法在 35–69% 的事件中找到本可达成目标的动作,而知道模拟器但不知结果的 oracle 规划器只有 14–23%。

这项工作把"区分坏决策与坏运气"变成了可计算的数学问题,并刻画了在不连续点、坐标折叠和缩放型扰动下的失效模式。对从事机器人控制、决策规划或基于世界模型的强化学习的开发者而言,它提供了一条无需收集反事实数据就能改造现有模型的可行路径。

原文 · arXiv cs.AI

Counterfactual World Models: JEPAs That Reason About "What If"

World models operate in a world they cannot fully observe. When a hidden disturbance, such as a current pushing a boat, changes the outcome of an action, a counterfactual question follows: what would have happened under a different action, subject to the same disturbance? Its answer separates bad decisions from bad luck. We prove that world models trained on factual transitions cannot answer it on their own, whatever their data or capacity. For many physical systems, the observed outcome and a simple property of the dynamics suffice: natura non facit saltus, nature makes no leaps. When actions and disturbances act as separate forces, their interaction is second-order in time, so the counterfactual outcome is, to first order, the observed outcome plus the difference between the world model's predictions under the alternative and factual actions. This yields a counterfactual world model with no counterfactual data, extra parameters, or retraining. For transitions of duration $h$, we prove an $O(h^2)$ bound on the counterfactual error relative to the disturbance variance, establish a matching minimax lower bound from unpaired factual data, and characterize failures at discontinuities, coordinate folds, and disturbances that scale the command. We validate these predictions from pixels with JEPA world models on frozen DINOv2 and masked autoencoder (MAE) encoders. On four rendered simulators, our approach improves counterfactual $R^2$ from $38$--$46\%$ to $86$--$96\%$ with the same world models; on a fifth, an underwater vehicle, it reaches $90$--$94\%$ with sixteen times more factual data. Four further frozen encoders, DINOv3, SigLIP~2, CLIP and EVA-02, give $81$--$95\%$ on the same simulators. In hindsight planning, it finds the action that would have reached the goal in $35$--$69\%$ of events, against $14$--$23\%$ for an oracle planner that knows the simulator but not the outcome.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:JEPA、DINOv2、世界模型、反事实推理,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Randall Balestriero, Yann LeCun, Kerda Varaku),经 AITOP 多维评分管线结构化提炼。