论文

REVE:复用编码器状态为大音频语言模型修正幻觉

REVE: Efficient Hallucination Correction for Large Audio-Language Models via Reused Encoder States

精选理由

给模型生成的声音事件描述做事实核查,不用再跑一遍音频编码,延迟只有 CED-Base 的十八分之一,音频方向的研究者可以看看。

大音频语言模型有时会生成输入中不存在的声音事件描述,传统做法需要额外的音频事件检测器和第二次音频编码。REVE 方法直接复用目标模型已计算出的编码器状态,用两个读出头提取类别分数并做类感知融合来验证生成结果。在 AudioSet 上,REVE 在忠实提及召回约束下去除了 92.9% 无标签支持的提及。其完整验证延迟约为 CED-Base 路径的 1/18,且新增参数更少,无需第二次音频编码。

原文 · arXiv cs.AI

REVE: Efficient Hallucination Correction for Large Audio-Language Models via Reused Encoder States

Large audio-language models may mention acoustic events that are absent from the input. A separate audio event detector can verify these mentions, but doing so requires a second audio encoder and a separate forward pass. We propose Reused Encoder States for Verifying Events (REVE), a lightweight method that uses states already computed by the target model. One readout summarizes class scores across audio frames, while another uses pooled states from four consecutive frame intervals. Class-aware score fusion combines their outputs to verify generated event mentions without encoding the audio again. On AudioSet, REVE removes 92.9% of label-unsupported mentions under a faithful-mention recall constraint. With fewer added parameters and no second audio-encoding pass, REVE achieves a reduction comparable to those of CED-Tiny and CED-Base. Its complete verification latency is about 1/18 of the CED-Base path. Results on controlled DESED mixtures and different target-model architectures further confirm the effectiveness of encoder-state reuse.