UniWALM 将视频、动作、语言统一进单一模型用于机器人操控
UniWALM: Unified World Action Language Model for Embodied Robot Control
一篇机器人方向的论文：UniWALM 把动作和语言记忆塞进一个模型一起生成，真实抓物成功率从 11% 拉到 61%，比 Mem-0 那套省事不少。
UniWALM 是一个统一的世界-动作-语言模型，在单一 mixture-of-transformers 主干中同时学习视频、动作和语言，三种模态互为条件信号与监督目标。它用同步协同去噪的扩散头在一次推理中同时解码动作、子任务和文本记忆，省去独立规划阶段，生成语言在下一步被重新消费，形成无需外部记忆库的语言即记忆闭环。训练上引入 Flow Matching with Correction（FMC），让模型学会从受扰动作轨迹恢复，推理时不增加开销。在 MolmoSpaces 组合式抓放任务和 RMBench 记忆子集上，UniWALM 超过同主干基线 FastWAM，在 RMBench 达到 34.5%，超过配备关键帧的 Mem-0 的 28.5%。在真实世界 Find-the-Item 任务上，文本记忆把成功率从 11.1% 提升到 61.1%，而现有 VLA 不超过 33.3%。
UniWALM: Unified World Action Language Model for Embodied Robot Control
Motivated by the human ability to integrate experience across language, perception, and action, we introduce UniWALM, a unified world-action-language model that learns video, action, and language in a single shared representation, where each modality is both a conditioning signal and a supervision target. We ask: can jointly generating language enrich this shared representation, improving manipulation and supporting memory-dependent behavior? Unlike prior approaches that attach a reasoning or vision-language model to a video-action backbone and run them as distinct stages, UniWALM unifies all three modalities inside a single mixture-of-transformers backbone whose per-modality diffusion heads co-denoise synchronously, decoding subtask and textual memory in the same pass as action with no separate planning stage; the generated language is re-consumed on the next step, closing a language-as-memory loop without an external planner or memory bank. To keep the action stream reliable under this richer objective, we further introduce Flow Matching with Correction (FMC), a training objective that teaches recovery from perturbed action trajectories at no inference-time cost. UniWALM consistently improves over same-backbone baselines (e.g., FastWAM) on compositional pick-and-place (MolmoSpaces) and across the memory-dependent subset of RMBench, where it reaches 34.5% without any pretrained language or reasoning model and exceeds Mem-0's keyframe-equipped setup (28.5%) using only a single anchor frame. On a real-world Find-the-Item task, generating textual memory lifts success from 11.1% to 61.1%, while state-of-the-art VLAs stay at or below chance (<=33.3%).
