AceAction 数据集发布：面向 3D 游戏环境的分层视觉-语言-动作学习
AceAction-5M: Hierarchical Goals for Generalist Action Learning Across 3D Game Worlds
一个拿 3D 游戏录像训练智能体的数据集，把任务拆成子目标再对齐键鼠操作，500 条轨迹审核通过率 95.8%，做 VLA 研究的可以看看。
AceAction 是一个跨多款 3D 游戏的人类操作演示数据集，用于训练智能体的技能组合与子目标切换能力。数据按完整任务组织，将总体指令、有序子目标和局部技能指令与画面观测、键鼠动作在时间轴上对齐。标注由视觉-语言模型流水线自动生成，能从交互对象变化中识别子目标，并把导航目标锚定到可辨识的路标点。演示覆盖导航、侦察、战斗、交互和恢复等场景，500 条轨迹的人工审核通过率达 95.8%。该数据集为在动态游戏环境中执行复杂指令的智能体研究提供了数据基础。
AceAction-5M: Hierarchical Goals for Generalist Action Learning Across 3D Game Worlds
Video games provide a practical setting for collecting vision-language-action (VLA) data, whose acquisition in physical environments is costly. Existing datasets, however, remain limited by restricted game coverage, sparse language supervision, or temporal hierarchies that do not necessarily reflect changes in task objectives. We introduce AceAction, a dataset of human gameplay across multiple 3D games for learning skill composition and subgoal transitions. AceAction organizes demonstrations around complete tasks, linking overall instructions, ordered subgoals, and local skill instructions to temporally aligned visual observations and keyboard-and-mouse actions. To construct these annotations, we develop a vision-language model pipeline that identifies subgoals from changes in objectives or interaction targets, anchors navigation goals to visually recognizable waypoints, and composes adjacent subgoals into task instructions. The resulting demonstrations cover navigation, reconnaissance, combat, interaction, and recovery, with explicit stage boundaries that connect task progression to fine-grained control. We assess annotation quality through model-based verification and human inspection. An audit of 500 trajectories yields a 95.8% pass rate across independent reviewer assessments. AceAction provides a data foundation for studying how agents execute, combine, and switch skills to follow complex instructions in dynamic game environments.