论文

研究区分驾驶VLA模型的指令遵循与轨迹预测

Instructions Before Actions: Prospective and Retrospective Language Supervision for Driving Vision-Language-Action Data

精选理由

做自动驾驶或VLA训练的值得看这篇。它用48条路线和87个机动动作证明轨迹误差降了23-59%不等于模型真的听懂指令,还对比了Alpamayo 2 Super和Qwen-Drive对指令的响应差异。

arXiv 论文《Instructions Before Actions》研究驾驶视觉-语言-动作(VLA)模型的监督数据收集方式,将导航指令分为行动前发出的 prospective 指令与观察行动后补写的 retrospective 标签。团队构建 ADVLAT-Engine 同步语音指令、道路场景视频和车辆轨迹,在 48 条重放路线上对比 Google Maps 与两名真实导航应用,并在 87 个同时被 Google Maps 和乘客播报的机动动作上发现乘客更依赖场景的语言表达。

正文

论文评估四个开源驾驶 VLA 在用 prospective 导航应用数据微调前后的表现:微调使乘客驾驶轨迹误差降低 23-59%,但指令影响不一致。在真实路口的反事实指令测试中,微调后的 Alpamayo 2 Super 在 15 米和 5 米处准确率分别提升 31 和 18 分,而 Qwen-Drive 用离散命令比用整句更可靠地跟随替代机动。作者还收集了四人对驾驶录像的 retrospective 标注,相关对比分析仍在进行中。

这项工作把机动预测与指令遵循区分开,为自动驾驶 VLA 的训练数据评估提供了比较 prospective 和 retrospective 监督的框架。

原文 · arXiv cs.AI

Instructions Before Actions: Prospective and Retrospective Language Supervision for Driving Vision-Language-Action Data

Driving vision-language-action (VLA) models can accept navigation instructions, yet predicting a driver's trajectory does not establish that a model follows the instruction. We study how driving-language supervision is collected and evaluated, distinguishing prospective instructions issued before an action from retrospective labels written after observing it. We develop ADVLAT-Engine to synchronize spoken instructions, road-scene video, and vehicle trajectories, and compare three navigation apps on 48 replayed routes. On 87 maneuvers announced by both Google Maps and a passenger, the passenger uses more scene-dependent language. We evaluate four open driving VLAs before and after fine-tuning on prospective navigation-app drives. Fine-tuning reduces passenger-drive trajectory error by 23-59%, but instruction effects remain inconsistent. At real intersections, counterfactual instructions reveal stronger responsiveness: fine-tuned Alpamayo~2 Super gains 31 and 18 accuracy points at 15 and 5~m, while Qwen-Drive follows alternative maneuvers more reliably with a discrete command than with a sentence. We additionally collected retrospective annotations from four people for the driving recordings and describe controlled comparisons of annotation content, timing, and downstream learning; those analyses are pending. Our findings separate maneuver prediction from instruction-following and establish a framework for comparing prospective and retrospective supervision.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:VLA、自动驾驶、Alpamayo 2、Qwen-Drive,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Angel Martinez-Sanchez, Ross Greer),经 AITOP 多维评分管线结构化提炼。