NAES 方法让冻结 CLIP 正确处理否定描述:COCO-MCQ 达 82.4%
Saying No by Asking Yes: Affirmative Decomposition and Evidence Reflection for Frozen CLIP
CLIP 这类模型一直分不清“有 X”和“没有 X”,这篇论文只加一个轻量分解器 NegSplit 就解决,不用微调模型。COCO-MCQ 提到 82.4%,比 KTN 高近 3 个点,还能直接用在 SigLIP 上。做图像检索或多模态匹配的开发者可以直接套用这套打分思路。
论文提出 Negation-Aware Evidence Scoring(NAES),针对冻结视觉语言模型无法正确处理否定描述的问题:查询“没有烤面包机的厨房”反而会偏好含烤面包机的图片。方法核心是利用模型擅长肯定描述的不对称性,将否定描述重构为肯定查询再组合证据。
具体流程分两步:轻量文本分解器 NegSplit 提取被否定的片段和剩余的肯定上下文。若存在肯定剩余内容,NAES 匹配该内容并对否定对象施加基于存在性的修正;若没有肯定内容,则查询被否定对象的存在情况,围绕参考点对证据进行反射式评分。
实验结果:使用 LAION-2B CLIP 在 COCO-MCQ 达 82.4%、VOC2007-MCQ 达 93.0%,超过 KTN 2.9 和 0.9 个百分点;使用 OpenAI CLIP 时超过 PeakPatch 4.4 和 26.0 个百分点。方法还提升了 COCO 和 MSR-VTT 上的否定描述检索效果,且适用于 SigLIP、SigLIP 2 及更大的 CLIP 编码器。
Saying No by Asking Yes: Affirmative Decomposition and Evidence Reflection for Frozen CLIP
Frozen vision-language models often match concepts mentioned in a caption without respecting their negation: a query "a kitchen with no toaster" favors an image containing a toaster. Yet these models are substantially stronger at matching affirmative descriptions. We exploit this asymmetry with Negation-Aware Evidence Scoring (NAES), which reformulates negated descriptions into affirmative queries and combines their evidence by caption structure. A lightweight text decomposer, NegSplit, identifies this structure by extracting negated spans and the remaining affirmative context. For captions with an affirmative remainder, NAES matches that content and applies a presence-based correction for the denial. When no affirmative content remains, it queries the denied object's presence and reflects the evidence around a reference to score the negated description. With LAION-2B CLIP, NAES achieves 82.4% accuracy on COCO-MCQ and 93.0% on VOC2007-MCQ, surpassing KTN by 2.9 and 0.9 percentage points; with OpenAI CLIP, it exceeds PeakPatch accuracy by 4.4 and 26.0 points, respectively. NAES also improves negated-description retrieval on COCO and MSR-VTT, with gains extending to SigLIP, SigLIP 2, and larger CLIP encoders. The strongest multiple-choice results use competing descriptions to center and align evidence; retrieval improvements persist without this conditioning. These findings show that explicitly separating affirmative matching from denied-object evidence can substantially improve negation-aware ranking.
本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:CLIP、SigLIP、NegSplit、多模态,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始信息发布自一手权威信源「arXiv: OpenAI」(Tier-1 一手发布),具备高确定性参考价值。截至目前,全网已有 11 家独立信源对该事件进行交叉印证与跟进报道(含 Meta AI、Shubham Saboo、Decoder 等),属于近期高关注度行业信号。