研究:偏好优化让视觉语言模型少幻觉却漏掉更多物体
Say Less, Hallucinate Less? Diagnosing and Restoring Object Coverage in Preference-Optimized Vision-Language Models
这篇论文指出偏好优化压幻觉的真相:很多模型只是变相提前结束生成,物体漏说了一半还没人发现。作者还给出免训练的修复方法,做视觉模型评测或后训练的人应该看看。
一篇新论文发现,四个基于 LLaVA-1.5-7B 的公开偏好优化(PO)检查点在降低幻觉的同时也降低了物体覆盖率,输出提及的物体类别更少,在 CHAIR 上的集合级 F1 没有明显提升。
分析表明,仅给基础模型加正向 EOS logit 偏置提前停止生成,就能在两个不重叠的 COCO 集合上复现三个检查点 58%–86% 的 CHAIRi 增益,说明所谓幻觉减少很大程度只是“说得更少”。论文还提出免训练的 base-guided 物体类别恢复方法:把基础模型高置信度输出的类别追加到 PO 描述中,三个检查点在保留 COCO 集上的召回率与基础模型差距缩小到 1 个点以内,且幻觉率更低。
作者将开源召回率受控评测的代码。
Say Less, Hallucinate Less? Diagnosing and Restoring Object Coverage in Preference-Optimized Vision-Language Models
Preference optimization (PO) is the standard way to reduce object hallucination in large vision-language models. We find that four public PO checkpoints of LLaVA-1.5-7B also lower object coverage. They name fewer object categories, and their lower hallucination rates bring no significant gain in set-level object F1 on CHAIR. We therefore ask how much of the PO gain the base model can reproduce by stopping earlier. A positive end-of-sequence (EOS) logit bias lets the base model reach the recall of each PO model, and this early stopping alone reproduces 58 to 86 percent of the CHAIRi gain of three checkpoints on two disjoint COCO sets. Writing longer does not bring the lost categories back. Under greedy decoding, a negative EOS bias recovers only 2 to 9 percent of the lost CHAIR recall of these checkpoints, and 94 to 96 percent of the added object mentions repeat categories already named. On CHAIR, the base caption of the same image still names about half of the missed object categories. We therefore propose base-guided object-category restoration, which appends high-confidence base-only categories to the PO caption without training. With thresholds fixed on a development set, it brings three checkpoints within one point of the base recall on a held-out COCO set with a lower distinct-object hallucination rate than the base model. Against the recall-matched base stopping curve, its gain holds for RLAIF-V near the base recall and for RLAIF-V and CHAIR-DPO at an intermediate recall. We will release the code of the recall-controlled evaluation.
本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:LLaVA-1.5、CHAIR、幻觉、多模态,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Hansang Lee),经 AITOP 多维评分管线结构化提炼。