论文

GeoState:视图解耦几何表征用于 3D 世界建模

GeoState: Learning View-Decoupled Geometry States for 3D World Modeling

精选理由

这个叫 GeoState 的方法把相机运动从场景变化里拆开,3D 世界模型在 Waymo 上效果更好,还能零样本跑 KITTI 和 nuScenes。

GeoState 是一种面向预测式 3D 世界建模的视图解耦几何表征,把场景状态演化与相机运动分离开。它通过 scene tokenizer 将多视图 GFM 特征聚合为视图解耦的场景状态,再由自回归世界模型预测未来状态,相机运动单独建模。在 Waymo 数据集上超越此前 3D 世界模型,并以零样本方式泛化到 KITTI 和 nuScenes,且整个 rollout 全程保持单一上下文推导的尺度,无需逐帧尺度对齐。

原文 · arXiv cs.LG

GeoState: Learning View-Decoupled Geometry States for 3D World Modeling

Recent 3D world models increasingly use features from geometry foundation models (GFMs) as predictive world states, benefiting from their rich geometric priors. However, the view dependence of these features entangles scene dynamics with camera-induced variation, making the resulting state evolution harder to predict. We propose GeoState, a view-decoupled geometry representation for predictive 3D world modeling that separates scene-state evolution from camera-dependent observation. Specifically, a scene tokenizer aggregates multi-view GFM features into a view-decoupled scene state, which a geometry decoder queries using camera rays. An autoregressive world model then predicts future scene states, while camera motion is modeled separately. The predicted states can therefore be decoded under either predicted or externally specified cameras, while retaining a single context-derived scale throughout the rollout without per-frame scale alignment. Experiments show that GeoState outperforms prior 3D world models on Waymo, supports geometrically consistent cross-view querying, and generalizes zero-shot to KITTI and nuScenes. Project page: https://hongchang-liu.github.io/GeoState-project-page/