论文

Concat IDM:用动作可预测性改进潜在世界模型

Learning Intrinsic Dynamics in Latent World Models

精选理由

一篇关于世界模型的新论文,提出的 Concat IDM 在 Push-T 任务上把成功率从 9.1% 拉到 77.2%,做机器人规划或强化学习的可以看看思路。

arXiv 论文研究了世界模型中表示与动作可预测性之间的关系,核心假设是让模型从连续潜在状态重建动作,能防止表示丢失动作与后果之间的区别。理论上,论文推导了当编码器合并不同动作的可区分结果时,平方逆预测误差存在正下界。实验中提出的 Concat IDM 从拼接的潜在状态预测动作,并利用离线轨迹中已有的动作标签更新共享编码器。在 Push-T 任务、24 种外观变化的设置下,Concat IDM 将规划成功率从 9.1% 提升到 77.2%,且配对改进比单一外观设置高出 52.9 个百分点。

原文 · arXiv cs.AI

Learning Intrinsic Dynamics in Latent World Models

Learning intrinsic dynamics---the relationship between actions and their consequences---is essential for world models. However, visual variations unrelated to the underlying control process can obscure this relationship in learned representations. We investigate the connection between action predictability, defined as the ability to reconstruct an action from consecutive latent states, and the preservation of intrinsic dynamics. Our central hypothesis is that encouraging action predictability discourages representations from discarding the distinctions between different actions and their consequences. We provide two complementary forms of evidence for this connection. Theoretically, we derive a positive lower bound on squared inverse prediction error when the encoder merges distinguishable outcomes of different actions from the same state and appearance, and show that forward prediction can still be exact on those transitions. Empirically, controlled appearance-diversity comparisons, supervision interventions, and latent probes support the role of action prediction in preserving control-relevant transition information. We instantiate action predictability with Concat IDM, which predicts actions from concatenated latent states and updates the shared encoder using action labels already available in offline trajectories, without changing the test-time planner. Across three independent training runs on Push-T with 24 appearances, Concat IDM increases planning success from 9.1% to 77.2%; its paired improvement is 52.9 percentage points larger than that observed with a single appearance. Together, these findings support action predictability as an inductive bias toward intrinsic dynamics and demonstrate the practical benefits of Concat IDM for latent planning under visual diversity.