PhysiCalWorld:通过上下文校准让世界模型感知场景物理属性
PhysiCalWorld: In-Context Physics Calibration for World Models
一篇世界模型论文,思路挺巧:不微调模型,只用一段探索交互编码成物理描述符,就能让预测贴合场景真实物理,规划成功率提升明显。
arXiv 论文 PhysiCalWorld 提出用上下文提示(in-context prompting)校准动作条件世界模型,无需微调或解析式物理建模。方法将测试场景中采集的一段探索性交互视频编码为紧凑的物理描述符 token,用于锚定后续预测,使其贴合场景真实物理行为(如质量分布、关节阻尼、表面摩擦)。该模型仅用 latent prediction loss 端到端训练,不需要物理标签;推理时描述符只需提取一次,可在多次 rollout 中复用,计算开销可忽略。在覆盖多种物理变化的四个环境中,PhysiCalWorld 明显提升了 rollout 预测精度和视觉规划成功率,学习到的描述符还能泛化到训练时未见过的物理参数范围。
PhysiCalWorld: In-Context Physics Calibration for World Models
Existing world models typically perceive a scene through current visual frame or a narrow history window. However, these observations may not reveal the physics properties that determine transition dynamics, such as an object's mass distribution, joint damping, or surface friction. Without calibrating these physics factors for world models, predictions tend to regress toward an empirical prior and induce severe drift in novel test scenes. To this end, we propose PhysiCalWorld, which calibrates action-conditioned world models to the instance-specific dynamics via in-context prompting, without fine-tuning or analytical physics modeling. Our method encodes an exploratory interaction clip collected within the test scene into a compact physics descriptor token, which grounds subsequent predictions in the scene's actual physical behavior. PhysiCalWorld is trained end-to-end with the latent prediction loss alone, without physics labels. At inference time, the descriptor token is extracted once and can be reused across rollouts with negligible compute overhead. We evaluate PhysiCalWorld across four environments covering diverse physics variations and show that it substantially improves rollout prediction accuracy and visual planning success rate. The learned descriptors also form a physically structured latent space and generalize beyond the physics ranges seen during training.