研究揭示视觉语言模型抗干扰能力与提示词冗长的关系
Cross-Modal Attention Acts as a Frequency Filter: Why Verbose Prompts Improve Robustness in Vision-Language Models
这个研究挺有意思的，它发现给AI模型问问题的时候，用更详细、更啰嗦的描述，能让模型在处理有问题的图片时更不容易出错，比如把‘有猫吗？’改成‘请仔细看并回答：有猫吗？’。
研究发现，当问题表述更详细时，视觉语言模型（如Qwen3-VL和LLaVA-OneVision）在图像受损时的表现会更稳定。例如，将“Is there a cat?”改为“Please look carefully and answer: is there a cat?”，模型对图像噪声的敏感性会降低。这种差异源于问题条件下的跨模态注意力机制，它会像频率滤波器一样处理图像信息，冗长的问题会拓宽其频率响应范围。
Cross-Modal Attention Acts as a Frequency Filter: Why Verbose Prompts Improve Robustness in Vision-Language Models
Vision-language models (VLMs) are fragile under image corruption. We find that the wording of the question affects VLMs in two opposite ways. Verbose questions make VLMs substantially more robust---e.g., rephrasing "Is there a cat?" into "Please look carefully and answer: is there a cat?". Conversely, VLMs become more fragile under corruption when the question is semantically complex or finer-grained, e.g., "what colour is the cup left of the chair?" instead of "is there a cup?". Both effects stem from question-conditioned cross-modal attention, which induces a spectral filter over image patches: verbose questions broaden its frequency support, while fine-grained questions concentrate it onto fewer visual scales. The model's answer drifts most when this filter and the corruption sit on the same spatial frequencies. We test the filter view on Qwen3-VL and LLaVA-OneVision across GQA and CLEVR; verbose paraphrasing reduces drift variance by 70--81% on the 8B models. The practical recipe---pad the prompt---further yields measurable gains in accuracy, even under image corruption.
