产品

LlamaIndex 发布 ParseBench-2 文档 OCR 基准,回应同行质疑

精选理由

选 OCR 工具的人可以看下 ParseBench-2,榜单全开放,数据集和评分标准都在 GitHub 上可查。它把 OCR 失败案例做了细分分类,比各家厂商自己发布的"自己第一"榜单更透明,选型前对比一下挺有用。

LlamaIndex 创始人 Jerry Liu 介绍上周发布的 ParseBench-2,这是他们对各类文档 OCR 工具最全面的基准测试,已在 parsebench.ai 上线,同时附带详细的技术报告。榜单完全开放:没有隐藏的评审提示词和测试数据集,并通过 GitHub/Hugging Face 的提交持续更新排行榜。

正文

该基准发布后在文档 OCR 厂商中引发争议。有厂商批评这是"营销噱头",随后自己发布低质量的基准;另有厂商称"每家在自己的榜单上都是第一",也随即推出自己排名第一的基准。Jerry Liu 回应称他们在部分维度上并非第一,其公平性努力未获认可。

团队吸收了建设性反馈,正在迭代新版本。即便不采纳其解析方案或总分,测试对不同失败类型的细分分类体系仍有参考价值,可作为选型文档 OCR 方案的参考资源。

原文 · Jerry Liu

Last week we released ParseBench-2, our most comprehensive effort to benchmark various document OCR tools. It drew quite a bit of ire among our peer document OCR vendors, who offered their own reactions: ---- “This is marketing slop” - vendor 1 who immediately goes and publishes lower-effort marketing slop (*cough benchbench cough*) ---- “everybody is number 1 on their benchmark” - vendor 2 who immediately shows their own benchmark where they’re #1 (also we’re not even #1 on some dimensions! No one actually cared about our efforts to be more fair) there was some constructive feedback which we’re taking into account and iterating on a new release. But it was mostly masked by some (pretty funny) noise and posturing. Unlike other vendors, our evals are completely open - no hidden judge prompt, no hidden test dataset, and we actively update the leaderboard from Github/Hugging Face submissions. If you’re in the market for document OCR, I think ParseBench-2 is probably the most comprehensive resource for comparing different solutions. Even if you completely ignore our parsing solutions or overall score, there’s value in understanding the detailed taxonomy we’ve created over different failure types. Come check it out! parsebench.ai Also our tech report is extremely detailed and the result of incredible effort from our research team. Come check it out: parsebench.ai/parsebench-2.p…📊 10 ⚡

产业坐标与信源核验

本条情报归属 AITOP「产品」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」「中美大模型技术代差与商业化对照」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:ParseBench-2、LlamaIndex、OCR、文档解析,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「Jerry Liu」(@jerryjliu0),经 AITOP 多维评分管线结构化提炼。