Artificial Analysis 改用 Time per Task 作为 API 性能评测头号指标
评测机构 Artificial Analysis 把指标从每 token 速度换成整个任务耗时，还有默认档位的耗时和成本对比，选模型前看看能省不少试错时间。
Artificial Analysis 对覆盖约 1000 个 serverless 端点的每日速度、延迟和缓存评测进行更新，头号指标改为 Time per Task，衡量模型完成一个任务的总耗时而非流式速度。End-to-End Output Speed 改为统计从发出请求到收到最后一个 token 的全流程，输出 token 数改用各厂商 API 报告值。评测加入多轮编程与知识工作的智能体负载，头号数据取过去 5 天平均值，并新增各实验室 API 默认 effort 档位的对比。数据显示 Claude Opus 5.5 从 max 档的约 26 分钟、10.04 美元每任务降到 medium 档的约 6.8 分钟、2.45 美元。
Announcing the next version of Artificial Analysis API Performance Benchmarking: Time per Task is now our headline performance metric, measuring how long a model takes to complete a task rather than how fast it streams
The update applies to the ~1,000 serverless endpoints we benchmark multiple times a day for speed, latency and caching.
Key changes: ➤ Time per Task is now the headline metric to compare models: as workloads become more agentic, the time to complete a task matters more than speed per token. A model that is fast per token but verbose can take longer than a slower model that uses fewer tokens. Time per Task divides a model's output tokens per Intelligence Index task by its End-to-End Output Speed.
➤ End-to-End Output Speed covers the full request: speed is now measured from sending the request to receiving the final token. That includes the wait before the first token (prefill, and reasoning for models that hide it) as well as generation. Output tokens now come from each provider's API-reported counts instead of the o200k_base tokenizer, and we monitor for discrepancies in reported counts.
➤ Agentic workloads are now part of every measurement: we benchmark short, document-length and long-context prompts, plus multi-turn sessions across coding and knowledge work. Headline figures are the average across these workloads over the past 5 days.
➤ API default effort is now tracked alongside highest effort: most developers run models at the effort level each lab sets as its API default, so we now benchmark key models at that setting as well as at their highest effort. Effort names are not comparable across labs, so we use each lab's own default rather than one shared setting.
Key takeaways: ➤ OpenAI models make up most of the Intelligence vs. Time per Task Pareto frontier despite slower output speeds: across effort levels, GPT-6 Astra and GPT-6.1 Sol hold the frontier from ~45 to ~55 on the Intelligence Index. At max effort, GPT-6 Astra runs at 51 tokens/s but finishes a task in ~17 minutes, against ~26 for Claude Opus 5.5 (max, 111 tokens/s) and ~29 minutes for Gemini 4 Carbon (high, 71 tokens/s), because it uses ~50k output tokens per task against ~120k to ~170k.
➤ API default effort cuts Time per Task by ~3 to 5x and Cost per Task by ~2 to 4x for GPT-6 Astra, GPT-6.1 Sol and Claude Opus 5.5: all three default to medium effort, which scores ~5 to 11 points lower on the Intelligence Index in exchange. Claude Opus 5.5 falls from ~26 minutes and $10.04 per task at max to ~6.8 minutes and $2.45 at medium effort.
➤ Endpoints that optimize for decode speed alone see the largest drop in End-to-End Output Speed: high decode speeds can come at the cost of latency, for example by queuing requests to run at lower batch sizes, or by holding output and sending it in a single burst. Both lengthen the wait before the first token, which End-to-End Output Speed now counts.
