论文

SpatialASM 训练框架提升 LVLM 主动空间心理化能力

SpatialASM: Internalizing Active Mentalization for Spatial Reasoning

精选理由

一篇教视觉模型像人一样主动换视角做空间想象的研究,7 个基准上 4 种骨干模型都有提升,做空间推理方向的可以看看。

arXiv 论文提出 Active Spatial Mentalization(ASM),指 LVLM 在空间推理中主动切换视角并重建场景布局的能力。SpatialASM 框架用 allocentric 新视角生成器将假设视角锚定在视觉证据上,产出显式 ASM 轨迹和自动构造的视角转换任务两类监督信号。在覆盖感知到跨视角推理的 7 个空间基准上,该方法在 4 种 LVLM 骨干上取得空间推理提升,同时保留原有通用视觉能力,并为空间 CoT 监督稀缺问题提供可扩展的数据合成路径。

原文 · arXiv cs.AI

SpatialASM: Internalizing Active Mentalization for Spatial Reasoning

When solving complex spatial reasoning tasks, ordinary humans naturally revisit the scenes from task-relevant viewpoints and mentally reconstruct the spatial layouts from those viewpoints. But can Large Vision-Language Models (LVLMs) proactively think in the same way? We term this ability Active Spatial Mentalization (ASM), an intrinsic component of visual intelligence that remains substantially underdeveloped in current LVLMs. To bridge this gap, we introduce SpatialASM, a scalable framework to internalize ASM into LVLMs through training-time spatial simulation. SpatialASM employs an allocentric novel-view generator to ground hypothetical viewpoints in visual evidence and produces two complementary forms of supervision: i) explicit ASM trajectories that teach LVLMs when and how to adaptively invoke view planning and imagination during spatial reasoning, and ii) automatically-constructed perspective-taking tasks that further strengthens viewpoint-conditioned scene imagination. On seven benchmarks spanning relatively perception-oriented spatial understanding to more challenging tasks requiring multi-image, cross-view and viewpoint-transformation reasoning, SpatialASM yields performance gain on LVLMs of four LVLM backbones in spatial reasoning while retaining their original general vision capabilities. The experimental results demonstrate that our approach improves average accuracy across the seven spatial benchmarks. And the synthesis of ASM trajectories provides a scalable pathway to alleviate the scarcity of spatial Chain-of-Thought (CoT) supervision for eliciting active viewpoint exploration and mental scene simulation.