LabOPBench 发布：756 道实验决策题，最强多模态模型仅得 45.8%
LabOPBench: A Multimodal Benchmark for Evidence-Grounded Laboratory Decision Making
拿 756 道真实实验决策题考了 8 个多模态模型，最高分才 45.8%，还比不过博士生，看看模型到底错在哪。
LabOPBench 是一个面向实验室决策的多模态基准，包含 756 道由专家撰写并独立评审的题目，覆盖有机合成和材料科学。每道题结合实验证据、实验流程和开放式判断，使用题目专属评分细则打分。八个多模态模型的总体归一化得分在 28.8% 到 45.8% 之间。在 100 题的对照子集上，最强模型平均超过本科生，但低于博士生的平均表现。专家分析发现证据误读和证据-决策冲突是三个头部模型的主要错误类型。
LabOPBench: A Multimodal Benchmark for Evidence-Grounded Laboratory Decision Making
Laboratory experiments involve more than following predefined protocols; they require adaptive decision-making as the experimental state changes. Existing scientific benchmarks rarely assess such evidence-based decisions in laboratory settings. Here we present LabOPBench, a multimodal benchmark of 756 expert-authored, independently reviewed questions in organic synthesis and materials science. Each item pairs laboratory evidence with procedural context and an open-ended judgment, evaluated using a question-specific scoring rubric. Eight multimodal models obtain overall normalized scores of 28.8-45.8%, leaving substantial room for improvement. On a controlled 100-question subset, the strongest models outperform undergraduate participants on average but remain below the average performance of PhD participants. Expert analysis of responses from the three leading models identifies Evidence Misreading and Evidence-Decision Conflict as the dominant error patterns. These results reveal that current models still struggle to turn multimodal perception into reliable laboratory decision-making.
