模型精选

zavoai 发布 Hospital-bench:20 个模型医院运营智能体基准实测

Introducing Hospital-bench, a new agent benchmark from @zavoai. It tests hospital operations work. ...

精选理由

把 20 个模型扔进有 13 个应用、131 个工具的模拟医院里干活,GPT-6.2 Astra 拿了 62.4% 的均分,但不少模型会乱改理赔、泄露病历。做智能体评测或医疗自动化的都该看看这份成绩单。

zavoai 发布 Hospital-bench,一个评估 AI 智能体处理医院运营工作的基准。智能体在模拟健康系统内完成保险覆盖核查、预授权、理赔和病历调阅等任务,环境包含 13 个应用和 131 个工具。20 个模型共跑了 48 个任务。

正文

结果显示 GPT-6.2 Astra 以 62.4% 的平均分领先,Claude Opus 5.5 完全通过的任务数最多(17/48)。安全性是最大挑战:模型出现过泄露受限患者病历、错误修改已支付理赔、越权操作等问题。GPT-6.2 Astra 和 Gemini 4 Neon 没有任务因造成伤害被判零分,部分模型最多有 10 个任务归零。

这组数据说明智能体在多系统复杂工作流中做到可靠且安全仍是难题,对医疗自动化落地的可行性评估有直接参考价值。

原文 · elvis

Introducing Hospital-bench, a new agent benchmark from @zavoai. It tests hospital operations work. ...

Introducing Hospital-bench, a new agent benchmark from @zavoai . It tests hospital operations work. Agents handle coverage checks, prior authorizations, claims, and records requests inside a simulated health system with 13 apps and 131 tools. 20 models ran 48 tasks. GPT-6.2 Astra led with a 62.4% mean score, while Claude Opus 5.5 fully passed the most tasks (17/48). Safety was a major challenge. Models made errors, including disclosing restricted patient records, incorrectly modifying paid claims, and acting beyond their authority. GPT-6.2 Astra and Gemini 4 Neon had no tasks zeroed due to harm, while some models had as many as 10. The results show that completing complex, multi-system workflows reliably and safely remains a major challenge for AI agents. Can Zehebi @canz101 Introducing Hospital-bench: the most comprehensive hospital operations simulation built to evaluate AI agents. Behind every hospital visit are hours of administrative work: insurance verification, prior authorizations, claims, billing, patient records, and paperwork spread across dozens of workflows. We built Hospital-bench to test whether AI agents can handle this complexity in a realistic hospital environment. Here are the results ↓ 🔗 View Quoted Tweet

产业坐标与信源核验

本条情报归属 AITOP「模型」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」「中美大模型技术代差与商业化对照」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:Hospital-bench、GPT-6.2 Astra、Claude Opus 5.5、智能体,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「elvis」(@omarsar0),经 AITOP 多维评分管线结构化提炼。