LlamaIndex 发布 ParseBench-2 文档解析基准,3 天下载量破 3 万
做 RAG 或文档处理的可以看看,LlamaIndex 这套基准把 330 多种解析错误按严重度打分,比 OmniDocBench 和 LLM 评委那套更贴近人工判断。顺手还发了 OCRia-1.5 模型,表格表单解析明显升级,已经接入 LlamaParse 可以直接试用。
LlamaIndex 发布 ParseBench-2,定位为最细粒度的文档解析评估基准,上线 3 天月下载量达 30k,登上 Hugging Face trending 榜第 3。基准覆盖 4.3k+ 文档页,比 ParseBench-1 翻倍,并新增表单(forms)维度,对 330+ 种错误类型逐一标注严重度分数,再据此校准每项指标,使分数与人工判断对齐。
与现有基准相比,ParseBench-2 避开了两条路线的缺陷:OmniDocBench 依赖脆弱的确定性指标,ParseArena 的 LLM-as-a-judge 则粗糙、随机且受提示词和模型偏差影响。上一代 ParseBench 已有 50k+ 月下载量、测评过 150+ 个 OCR 方案,被 NVIDIA、Cohere、Perplexity、Aleph Alpha 采用,发布后团队已合并数十个社区 PR,持续修复标注和指标问题。
同期发布的还有 OCRia-1.5 文档解析模型家族,驱动 LlamaParse 全部解析模式,在复杂表格、表单、图表、版式和 bounding box 标注上达到当前最优水平。对需要在生产环境做文档处理的团队来说,可以按需挑选关心的指标子集直接接入。
ParseBench-2 hit 30k monthly downloads in 3 days, and is #3 on @huggingface trending benchmarks! 📈 In ParseBench-2, we tried hard to create a comprehensive taxonomy of metrics to evaluate document processing, while creating a severity-based framework to align these metrics with human judgment. Whether you’re deeply familiar with document OCR and document OCR benchmarks, or just getting started, we encourage you to check out the ParseBench-2 methodology. We’re happy to help you pick whatever subset of metrics you care about if you’re looking to process documents in production. We’ve merged dozens of PRs since the release date, including fixes/metric updates from some of our peers. We’re aggressively fixing any lingering annotation and metric issues. Report: llamaindex.ai/files/parseben… e Hugging Face: huggingface.co/datasets/llama… d Github: github.com/run-llama/Pars… f Jerry Liu @jerryjliu0 Today we’re introducing ParseBench-2 - our most ambitious document parsing benchmark to date. It is the only benchmark that provides both extreme granularity into every single type of production parsing error, while also making sure that metric scores are highly aligned with human judgment. We do this by painstakingly assigning severity scores to 330+ error types, and tuning each metric to match the severity scores. Existing benchmarks either use brittle deterministic metrics (e.g. OmniDocBench) or LLM-as-a-judge (ParseArena), which can help recognize equivalent representations but are coarse/stochastic/subject to bias by the prompt+model. ParseBench-2 also contains: ✅ 4.3k+ document pages (double from ParseBench-1). ✅ An extra dimension (forms) ✅ Much better ground-truth across all dimensions compared to ParseBench-1. This is the next evolution of ParseBench, our hugely popular benchmark that has 50k+ monthly downloads, over 150+ OCR solutions benchmarked, and has been adopted by companies like NVIDIA, Cohere, Aleph Alpha, Perplexity, and others. Alongside ParseBench-2, we are also releasing OCRia-1.5 - a new family of document parsing models that powers all our modes in LlamaParse. They represent a massive step up in capabilities from our latest release, and are state of the art in parsing complex tables, forms, charts, formatting, and assigning bounding boxes to complex layouts. Come check out our blog post, technical report, and website. Blog: llamaindex.ai/blog/introduci… Technical Report: llamaindex.ai/files/parseben… Website: parsebench.ai If you’re interested in trying out our OCRia-1.5 capabilities, come sign up for LlamaParse: cloud.llamaindex.ai
本条情报归属 AITOP「模型」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」「中美大模型技术代差与商业化对照」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:LlamaIndex、ParseBench、OCRia、文档解析,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「Jerry Liu」(@jerryjliu0),经 AITOP 多维评分管线结构化提炼。