DiaVLo：诊断视觉语言模型行为的因果分析框架
DiaVLo: Diagnosing Behaviours of Vision-Language Models
一个诊断 VLM 行为的框架，能标出模型哪里对齐、哪里跑偏，还给出因果估计找出影响最大的概念。
arXiv 论文提出 DiaVLo，一个诊断视觉语言模型（VLM）行为的框架，结合人工标注与 VLM 自身的生成能力，构建期望行为和实际行为的规格说明，暴露潜在失配问题。DiaVLo 还提供因果估计，定位对模型行为影响最大的概念。作者在多个开源 VLM 上、分类和生成两种条件下做评估，结果显示其行为标签与模型性能相关，并为性能数字提供解释背景。
DiaVLo: Diagnosing Behaviours of Vision-Language Models
Vision-language models (VLMs) rely on storing and transferring appropriate information across their sub-components. Verifying that the VLMs exhibit desired behaviours, while avoiding harmful ones, is central to their reliable deployment. Yet, methods that identify VLM behaviours remain scarce. We present DiaVLo, a diagnostic framework that leverages human curation and VLMs' generation capabilities to construct specifications of desired and observed VLM behaviours, surfacing potential misalignments. Beyond this, DiaVLo also provides causal estimates to identify the most influential concepts steering VLM behaviours. We evaluate DiaVLo on several open-source VLMs under both classification and generation conditions. Our experiments show that DiaVLo produces behaviour labels that correlate with model performance and provide context for measured performance. DiaVLo surfaced behaviours that are clearly aligned and misaligned, alongside patterns in how VLMs perceive, organise, and prioritise concepts.