LlamaIndex 发布 ParseBench-2：按错误严重度加权评估文档表格解析
做 RAG 或金融文档解析的朋友可以看看：LlamaIndex 新基准把表格解析错误分成 5 级严重度，数字翻号这种致命错误权重可以自己调到 25，比 OmniDocBench 那种编辑距离打分更能反映真实损失。
LlamaIndex 推出 ParseBench-2，用于解决文档解析中复杂表格难以量化评估的问题。该基准内置 330 多种错误类型，并将错误划分为 5 个严重度等级，其中最严重的一级包括数字错误、小数点错位、数值互换和列缺失；中等严重度包括表体行顺序重排、单元格被拆分等；格式差异和 HTML 标记差异则被归为无害错误。
技术机制上，ParseBench-2 允许用户通过 severity policy（严重度策略）对确定性指标进行加权调整。例如在金融场景中，丢数字或翻转正负号默认权重为 10，用户可自定义提高到 25，从而让指标能区分致命错误与普通偏差。
现有基准 OmniDocBench、OlmOCR-Bench 和 ParseBench-1 主要依赖编辑距离或二元测试的平均值，难以区分严重错误和轻微差异。对金融报表、RAG 知识库等依赖表格数据的下游应用来说，这种可自定义权重的评估方式更贴近真实业务损失。
One of the hardest parts around document parsing is parsing complex tables. Besides the task itself being difficult, it is also hard to measure what ‘good’ looks like. Ultimately it comes down to intuition around what is a catastrophic error for downstream knowledge work. * Formatting differences (markdown vs. html tags) are harmless * Making the header row a “data” row is also probably harmless (any human/model can interpret it as the header anyways) * Swapping the signs on a number, particular in financial reporting, is much more severe. It is hard to encode what “good” looks like in a quantifiable way. Existing deterministic tests/metrics (OmniDocBench/OlmOCR-Bench/ParseBench-1) use some variant of edit distance, or an average over binary tests, making it hard to distinguish severe errors from normal ones. With our recent release of ParseBench-2, our answer is to tune any deterministic metric by a severity policy, where you can explicitly state how good or bad different errors are. * For instance, in a finance setting, dropping digits/flipping signs is really bad! Our default severity weighting is 10, but you might decide 25 - the policy is meant to be customizable by the user. * Check out the examples below highlighting this. You can compare the impact of “actually bad” or “harmless” perturbations on a few different benchmarks incl. ParseBench-2. We have a taxonomy of 330+ error types, a big subset of which is defined around table errors, categorizing them from 5 severity levels: * most severe: wrong digits, decimals misplaced, values swapped, column missing * medium severe: body rows reordered, one cell split into two, etc. * harmless: formatting errors, html markup More details around table methodology in blog/tech report. Blog: llamaindex.ai/blog/introduci… Tech report: llamaindex.ai/files/parseben…
本条情报归属 AITOP「论文」核心栏目，重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」「中美大模型技术代差与商业化对照」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：LlamaIndex、ParseBench、OmniDocBench、RAG，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「Jerry Liu」（@jerryjliu0），经 AITOP 多维评分管线结构化提炼。
