AA-Robotics 基准首发：GPT-6 Astra 居机械臂操控榜首
GPT-6、Gemini 4、Claude 操控机械臂，GPT-6 Astra 夺冠，但比人慢约百倍。
Artificial Analysis 发布 AA-Robotics v0.1，让语言模型通过 HTTP API 直接控制 6 轴机械臂，完成抓取、空间推理和应变共 8 项零样本任务。GPT-6 Astra 以 69 分居首，GPT-6.1 Sol 得 65 分，Gemini 4 Argon 55 分，Claude Opus 5.5 拿下 49 分。成本差距明显：GPT-6.1 Sol 平均每任务 1.93 美元，GPT-6 Astra 高达 16.36 美元。目前模型平均每任务耗时约 30 分钟，比人类基线慢约 100 倍、贵约 10 倍，尚不具备生产可用性。
Welcome to the era of embodied AI.
We are launching AA-Robotics v0.1, our research preview that measures frontier language model performance on understanding, planning, and acting in the physical world. In v0.1, each model is given control of a 6-axis robot arm (5 joints and a gripper) and a simple objective. We’re releasing results for our first eight tasks today.
We find that frontier language models in coding agent harnesses have recently become capable of directly manipulating robot arms for simple tasks. Today, they are far too slow and expensive for any kind of production usage - nearly two orders of magnitude worse than human baselines. We expect this to change quickly.
GPT-6 Astra places first in AA-Robotics v0.1, followed by GPT-6.1 Sol, Gemini 4 Argon and Claude Opus 5.5.
AA-Robotics v0.1 details:
➤ Our first eight tasks test a range of capabilities: control and dexterity (precise grasping and object handling), spatial reasoning (how objects interact with one another) and adaptability (how models respond to the unexpected)
➤ Models get only brief instructions: how to connect to the robot arm (via a simple HTTP API), the arm's name and how to submit an attempt. They can view the scene through four cameras and write and run code to plan and send movements
➤ Once a model submits an attempt, its end state is graded against the task's pass criteria. A pass scores 100; an incomplete attempt earns partial credit. All attempts are zero-shot
Key findings:
➤ The GPT family tops the leaderboard: GPT-6 Astra scores 69, ahead of GPT-6.1 Sol at 65 and Gemini 4 Argon at 55. Claude Opus 5.5 (49) and Claude Fable 5.1 (46) trail
➤ Today’s models are incredibly slow at our tasks, taking ~30 minutes on average to complete each tabletop task
➤ Cost varies widely: GPT-6.1 Sol is the cheapest at $1.93 per task on average; GPT-6 Astra, the most expensive, costs $16.36, more than eight times as much
➤ Compared with a human, today's best-performing frontier language models are ~100x slower and ~10x more expensive
More results, clips, and how AA-Robotics v0.1 works follow in the thread below
