Artificial Analysis 联合 Google DeepMind 推出 MKA 多模态智能体基准
想看多模态智能体真实水平的话，这个 DeepMind 和 Artificial Analysis 合作的新基准挺扎实：200 个带图纸和扫描件的任务，Claude Fable 5.2 拿了第一但每任务要 32 美元，OpenAI 系模型便宜一个数量级还在性价比前沿，选型的人可以直接抄榜单。
Artificial Analysis 与 Google DeepMind 合作推出 Multimodal Knowledge Agent（MKA）基准，包含 200 个跨 10 个行业的私有真实视觉任务，要求智能体阅读建筑图纸、工程原理图、临床扫描等复杂视觉文件，产出幻灯片、表格、PDF 报告和标注图像等交付物。每项任务以约 70 条加权专家评分标准打分，分为基线 Regression 与更高难度的 Climb 两类，头条指标为 Climb WSR。
测试在开源智能体框架 Stirrup 中进行，配备代码执行、网页搜索、网页抓取与图像查看工具，最多 250 轮，并使用 GPT-6 Luna（medium）作为智能体评审。结果上 Claude Fable 5.2（max）以 57% 的 Climb WSR 居首，Gemini 4 Carbon（high）为 54%，Claude Opus 5.5（max）为 53%。
成本差异显著：Claude Fable 5.2 平均每任务 $31.97，约为 Gemini 4 Carbon（$6.29）的 5 倍、GPT-6.1 Sol（$1.11）的 29 倍，而 OpenAI 四款受测模型均处于性价比 Pareto 前沿。没有任何单一模型在全部 10 个行业领先，各垂直领域由 Claude、Gemini 系列分头占据第一。
We’ve partnered with Google DeepMind to launch their Multimodal Knowledge Agent (MKA) evaluation, a new benchmark for agentic knowledge work over complex visual files
@GoogleDeepMind’s MKA is a new benchmark requiring agents to review visually complex project files such as schematics and scans, and produce deliverables like slide decks, spreadsheets, reports, and marked-up images
Multimodal Knowledge Agent (MKA) overview:
➤ 200 private real-world visual tasks across 10 industries: Agents work from dense visual project files including architectural drawings, engineering schematics, and clinical scans to produce deliverables including slide decks, spreadsheets, PDFs, Word documents, and marked-up images. Unlike many agentic evaluations, these inputs emphasize realistic visual complexity and domain-specificity, and are genuinely required to complete the task rather than just providing additional context
➤ Climb vs. Regression rubric scoring: Each task is graded against expert rubrics, averaging ~70 weighted criteria per task. Criteria are split into ‘Regression’ (baseline checks models should pass) and ‘Climb’ criteria (harder checks designed to separate strong models). The headline score is Climb WSR (Weighted Satisfaction Rate) - the weighted proportion Climb rubric criteria successfully completed
➤ Evaluated in Stirrup with an agentic judge: We run every task in Stirrup, our open source agent harness, with code execution, web search, web fetch, and image-viewing tools (up to 250 turns), and grade submitted deliverables using GPT-6 Luna (medium) as an agentic judge
Key MKA results:
➤ Claude Fable 5.2 leads on multimodal knowledge work: Anthropic’s recently released Claude Fable 5.2 (max) leads the MKA leaderboard with a Climb WSR of 57%, followed by Gemini 4 Carbon (high) at 54% and Claude Opus 5.5 (max) at 53%
➤ Cost per Task spans nearly 30× within 10 points of the top score: Claude Fable 5.2 averages $31.97 per task, over 5× Gemini 4 Carbon ($6.29) for ~2 extra points, and ~29× GPT-6.1 Sol ($1.11). Below Carbon, all four OpenAI models tested sit on the cost-performance Pareto frontier (GPT-6 Astra, GPT-6.1 Sol, GPT-6.1 Terra, and GPT-6 Luna), with Claude Haiku 5.5 and MiMo-V2.6-Pro making up the rest of the frontier
➤ No single model leads across all 10 industry verticals: Claude Fable 5.2 leads five verticals (Software & Data, Engineering, Logistics & Mapping, Business, and Finance alongside Claude Sonnet 5.5), Claude Opus 5.5 (max) leads Education and Construction, Gemini 4 Argon (high) leads Science, Gemini 4 Carbon (high) leads Healthcare, and Claude Sonnet 5.5 (max) narrowly leads Law & Government
Thank you to the @GoogleDeepMind team for their work building this benchmark and collaborating with us to bring it exclusively to Artificial Analysis!
本条情报归属 AITOP「模型」核心栏目，重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中，该动态直接关联「中美大模型技术代差与商业化对照」「AI 开发者工具链、Coding Agent 与工程化实践」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：Google DeepMind、Claude Fable 5.2、Gemini 4 Carbon、多模态，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「Artificial Analysis」（@ArtificialAnlys），经 AITOP 多维评分管线结构化提炼。
