inkGround:面向手写笔迹恢复的视觉定位基准发布
inkGround: Benchmarking Fine-grained Visual Grounding Capabilities of Vision-Language Models for Cursive Handwriting in the Wild
arXiv 上这篇论文做了个新基准 inkGround,专门考 VLM 能不能从手写照片里恢复笔画顺序,Gemini-3.1-Pro 表现最好。
论文提出 inkGround 基准,任务是从整页相机拍摄的手写图像中恢复有序笔画轨迹和提笔/落笔状态。数据集包含 537 张页面图像,约 18.7k 个单词和约 163k 个笔画,参考轨迹由离线转在线模型生成,经 OCR 一致性过滤和人工核验。任务被定义为文本条件的点定位:模型需根据目标单词定位并输出有序笔画点。评测指标包括 DTW、Chamfer 距离等轨迹保真度,以及 Accurate Rate、Correct Rate 等渲染可读性。团队在 4 个开源和 3 个闭源 VLM 上测试,闭源模型如 Gemini-3.1-Pro 生成的轨迹最接近参考,排名在 IAM-OnDB 真实记录数据上同样成立。
inkGround: Benchmarking Fine-grained Visual Grounding Capabilities of Vision-Language Models for Cursive Handwriting in the Wild
Human handwriting is produced as an ordered sequence of pen movements, but an offline image keeps only the final ink. Handwriting trajectory recovery aims to restore the ordered stroke points and their pen-up/pen-down transitions. Existing image-trajectory pair datasets are recorded with digitizing tablets under controlled conditions, which limits writer coverage and stylistic diversity. Moreover, there is no benchmark that measures the visual grounding capabilities of general-purpose Vision-Language Models (VLMs) for the stroke trajectory recovery task. In this work, we introduce inkGround, a benchmark for stroke trajectory recovery from full-page, camera-captured handwriting. inkGround pairs 537 page images (~18.7k words, ~163k strokes) with reference trajectories produced by a pretrained offline-to-online model, filtered by OCR agreement, and verified by human raters. Because these references are pseudo-labels that represent one plausible pen trajectory, we also evaluate VLMs on the IAM-OnDB dataset, which provides recorded pen trajectories. We cast the task as text-conditioned point grounding: given a page image and a target word, the model must locate the word and output its ordered stroke points. Outputs are scored by trajectory fidelity (DTW, Chamfer distance), readability of the rendered strokes (Accurate Rate, Correct Rate), and style similarity. We benchmark four open-source and three frontier closed-source VLMs with a shared in-context prompt. Closed-source models, notably Gemini-3.1-Pro, produce legible trajectories close to the reference, and this ranking also holds on IAM-OnDB.