论文

研究提出计算机使用智能体可作为具身通用智能体

Computer-Use Agents Are Embodied Generalists

精选理由

同一套 CUA 什么都不改,换个软件界面就在 ECCV 四个比赛拿了名次,具身智能又省一个方向,挺有想法。

一篇 arXiv 论文提出,具身任务可以转化为与数字软件的交互:通过软件渲染物理环境的屏幕视图并提供键鼠控制接口。研究团队用同一套 CUA(计算机使用智能体)设计,仅更换软件,就在 ECCV 四个研讨会比赛中拿下第一、第二、第三和第六名的奖项,且未做任何针对比赛的智能体调优。论文认为 CUA 在软件上学到的视觉定位、精确点击和探索等技能可以直接迁移到具身任务上。

原文 · arXiv cs.LG

Computer-Use Agents Are Embodied Generalists

In many areas of AI, general-purpose models have increasingly replaced handcrafted methods and specialized architectures. We argue that embodied AI is poised for a similar shift. Our key claim is that embodied tasks can be formulated as interaction with digital software. Specifically, we build software that renders an agent's view of a physical environment on a screen and provides controls for acting in it. A computer-use agent (CUA) can then perform embodied tasks through the same screenshots and keyboard and mouse actions that it uses to operate ordinary software. Using a single CUA design and changing only the software, our entries win first-, second-, third-, and sixth-place prizes across four ECCV workshop competitions, without any competition-specific agent tuning. These results suggest that the skills CUAs learn on software, such as visual grounding, precise clicking, and exploration, already carry over to embodied tasks, giving CUAs a far broader scope in embodied AI than previously realized. We therefore advocate that computer-use agents can be embodied generalists. Further, we advocate that, by following the training strategy that made computer-use agents general, embodied AI can converge on a single general model as well, ushering in a new era of general embodied AI. Website: https://pranjal2041.github.io/cua-embodied