MLLM读不出人眯眼才见的文字?SquintBench与Squint OPSD方法
When MLLMs miss what humans read: diagnosing and learning to read squint text
多模态模型居然读不出人眯眼就能看到的隐藏文字,这篇论文造了个1076张图的测试集,还把成绩从26.5提到88.6,做法挺有意思。
论文提出 Squint Text 这一识别缺陷:多模态 LLM 无法读出图像粗结构中隐含、人类眯眼即可辨认的文字,并在生产审核流水线中发现了相关案例。作者构建了包含 1076 张生产环境审核图像的 SquintBench,以及用于机制诊断的合成数据集 SquintCtrl。分析显示隐层保留了粗信号但字形身份表示薄弱,给出正确空间区域后识别能力明显提升,说明空间选择是主要瓶颈。提出的 Squint OPSD 训练方法在 SquintBench 上达到 88.61 ANLS,未适配骨干模型仅 26.50,通用模型在原始视图下最好成绩为 24.44。
When MLLMs miss what humans read: diagnosing and learning to read squint text
Multimodal LLMs can fail to read text embedded in an image's coarse structure (text a human recovers by squinting) and report that no text is present. We name this class Squint Text and study it as a recognition deficiency, motivated by cases from a production moderation pipeline. We introduce SquintBench, comprising 1076 production-derived moderation images, and SquintCtrl, a controlled synthetic suite for mechanism diagnosis. Frozen-weight analysis shows that hidden text leaves a coarse signal while glyph identity is weakly represented; identity becomes substantially more accessible when the correct spatial regions are supplied, implicating spatial selection as a major bottleneck. Suppressing high-frequency content also improves reading at a fixed visual-token budget, while visual token count may also contribute. These findings motivate Squint OPSD: a frozen teacher receives a downscaled view and the ground-truth transcription as privileged context, and its token-level distributions supervise a student along on-policy trajectories. At inference, the student reads the original image alone. Squint OPSD reaches 88.61 ANLS on SquintBench, versus 26.50 for the unadapted backbone, 24.44 for the best general-purpose model on the original view, and 68.60 at best for an off-the-shelf model under a downscaled view. These results substantially narrow the gap between available visual evidence and its generated read-out without auxiliary views at inference.