研究揭示 Qwen-VL 与 InternVL3 的左右方向判断主要来自 ViT 位置编码
Where Do Vision-Language Models Get Left and Right? Position-Code Counterfactuals in Qwen2.5-VL, Qwen3-VL and InternVL3
做 VLM 空间理解评测或微调的人应该看看：这篇用镜像位置编码的反事实实验证明 Qwen2.5-VL、Qwen3-VL 和 InternVL3 的左右判断主要靠 ViT 位置编码，语言模型侧几乎不参与。结论直接指向关系绑定才是错误修复的正确靶点，比泛泛的评测报告有用。
一篇 arXiv 论文通过镜像 ViT 位置编码的反事实实验，定位了 Qwen2.5-VL-7B、Qwen3-VL-4B 和 InternVL3-2B 判断图像左右方向的来源。结果显示，仅镜像 ViT 宽度方向的位置编码，就能在 84.3-93.2% 的 CV-Bench 左右问答上复现翻转图像后的答案，其中 Qwen2.5-VL-3B 为 87.5%。这说明视觉侧位置编码是该能力的充分信号来源。
机制层面，在保持 ViT 编码不变的情况下，镜像语言模型侧的 token id 或 token 顺序对三个系列的左右答案影响很小，Qwen2.5-VL-7B 中该效应集中在全局注意力 ViT 块、且主要在最后几层。带定位目标的 CV-Bench 题目上，随机置换在颠倒物体顺序时比保持顺序多翻转 18.9-31.0 个百分点的答案；What'sUp 数据集上 on/under 关系比 left-right 更依赖内容与 token 顺序而非位置编码。
论文指出多数剩余错误与询问哪个物体无关，提示修复方向应在关系绑定、物体定位或参照物的使用上，而非简单调整位置编码。对从事多模态评测与可解释性的开发者有直接参考价值。
Where Do Vision-Language Models Get Left and Right? Position-Code Counterfactuals in Qwen2.5-VL, Qwen3-VL and InternVL3
A vision-language model may take left-right direction from the orientation of image content within patches, its vision transformer's (ViT) position code, or token positions and order in its language model (LLM). Ablations show the ViT code is necessary; we test sufficiency by mirroring only a position code. In Qwen2.5-VL-7B, Qwen3-VL-4B and InternVL3-2B, mirroring ViT width positions reproduces the flipped-image answer on 84.3-93.2% of CV-Bench left-right questions (Qwen2.5-VL-3B: 87.5%, partly a left fallback). With two options, this also counts disruption; a half-grid shift flips most answers too. With the ViT code intact, LLM-side id and token-order mirrors change few left-right answers in all three series (a weak id test in Qwen2.5-VL). In Qwen2.5-VL-7B the effect sits in the global-attention ViT blocks, mostly the last. On What'sUp, flipping content and token order, not the position code, keeps more on/under than left-right answers at the flipped-image answer. Post hoc analyses qualify these results. On CV-Bench items with located objects, random permutations flip 18.9-31.0 points more answers when reversing the objects' order than when keeping it. On What'sUp, in Qwen2.5-VL and Qwen3-VL, the share of the left-right changes made by random permutations that land on the mirror relation is mostly below chance. Counted as returns to the original answer, the What'sUp contrast of support with left-right relations is clear only in InternVL3-2B. Most remaining errors give the same relation whichever object is asked about, suggesting that repair may target relation binding, object localization or use of the reference object.
本条情报归属 AITOP「论文」核心栏目，重点聚焦中国 AI 产业与国产大模型生态。在跨区域产业观察坐标中，该动态直接关联「中美大模型技术代差与商业化对照」「AI 开发者工具链、Coding Agent 与工程化实践」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：Qwen2.5-VL、Qwen3-VL、InternVL3、多模态，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「arXiv cs.LG」（@Enhao Cui, Junzhi Yang, Yaoshun Lin, Yuhan Feng, Zijian Chen），经 AITOP 多维评分管线结构化提炼。