模型多源确认精选

医疗长文本基准对比:Anthropic 回答更全,OpenAI 更准更简洁

精选理由

Artificial Analysis 拿 14 个模型在两个医疗基准上掰头,结论挺反直觉:Claude 写得多反而更容易出错,GPT-6 综合分才最均衡。

Artificial Analysis 在 MLCR-AA 和 HealthBench Professional 两个医疗基准上对比了 14 个模型。Claude Sonnet 5.5、Opus 5.5 和 Fable 5.1 占据 MLCR-AA 完整性前三,比最好的 OpenAI 模型高出约 19-23 分,Opus 和 Sonnet 在 HealthBench Professional 原始分上也以 71.4% 和 70.8% 领先 GPT-6 Astra 的 69.7%。但 OpenAI 模型在准确性上占优:Anthropic 模型每个回答的篇幅约为 OpenAI 的 4-6 倍,准确性得分只有 19-61%,OpenAI 则为 89-97%。在同时奖励两者的综合分上,GPT-6 Astra 与 Claude Sonnet 5.5 并列 MLCR-AA 第一,并在 HealthBench Professional 长度调整分上小幅领先。

原文 · Artificial Analysis

On MLCR-AA and HealthBench Professional, Anthropic models give the most complete answers but OpenAI models are more accurate and concise

We compared 14 models with results on both MLCR-AA, our leaderboard for @wisedocsai's Medical Long Context Reasoning benchmark, and @OpenAI's HealthBench Professional. MLCR-AA grades answers about long medical records on completeness (covering every key point of the expert's answer) and accuracy (no claim contradicting the record), and its score requires both. HealthBench Professional grades replies to clinicians against physician-written rubrics: the raw score is the share of rubric points earned, while the length-adjusted score penalizes longer answers.

Key takeaways:

➤ Anthropic models lead on completeness and raw score: Claude Sonnet 5.5, Opus 5.5 and Fable 5.1 (all max with fallback) from @AnthropicAI hold the top three MLCR-AA completeness scores, ~19-23 points ahead of the best OpenAI model, and Opus and Sonnet post the highest HealthBench Professional raw scores (71.4% and 70.8%), narrowly ahead of GPT-6 Astra (69.7%)

➤ OpenAI models lead on accuracy and conciseness: OpenAI models hold the top MLCR-AA accuracy scores. Longer answers are less likely to be judged accurate, as each extra claim is another chance to contradict the record: Anthropic models write ~4-6x more per answer than OpenAI models, and score 19-61% on accuracy against 89-97%

➤ The gap closes on scores that reward both: GPT-6 Astra (max) ties Claude Sonnet 5.5 for #1 on MLCR-AA and narrowly leads the HealthBench Professional length-adjusted score, while Claude Opus 5.5 falls from #1 on the raw score to #6 length-adjusted

  • TestingCatalog10-21 18:57原文
  • ARC Prize10-21 19:30原文
  • Decoder10-20 16:40原文
  • Ethan Mollick10-20 21:25原文
  • IT之家10-20 23:45原文
  • arXiv: DeepSeek10-21 04:09原文
  • Tanishq Abraham (论文推介)10-21 04:28原文
  • swyx (AI Engineer)10-21 05:06原文
  • rohanpaul_ai10-21 07:56原文
  • @cb_doge10-20 10:59原文