论文

VeriSpeak 基准揭示语音事实核查短板:检索加推理达 86.1% 准确率

To Trust or Not to Trust: Retrieval-Augmented Fact Checking in Speech

精选理由

研究者做了个 3879 条语音声明的核查基准,发现 LALM 听到语音就变笨,加上推理才能到 86.1%,做语音事实核查的可以看看。

arXiv 论文提出 VeriSpeak 基准,包含 3,879 条覆盖时间、地理和关系类事实的语音声明,真假标签均衡。实验显示 LALM 在书面声明上核查可靠,换成语音形式时往往失败,存在明显的文本-语音模态差距。单独检索带来的提升有限,模型常把检索到的证据与语音声明本身混淆。检索结合显式推理后效果改善,经 thinking 微调的 LALM 达到 86.1% 准确率。数据集已在 Hugging Face 公开。

原文 · arXiv cs.AI

To Trust or Not to Trust: Retrieval-Augmented Fact Checking in Speech

Online misinformation increasingly appears in spoken formats such as news clips, podcasts, interviews, political speeches, and social media videos, creating a need for fact-checking systems that can verify claims directly from speech. We introduce VeriSpeak, a probe benchmark for studying speech-based fact verification in Large Audio Language Models (LALMs). VeriSpeak contains 3,879 spoken claims spanning temporal, geographical, and relational facts, with balanced true and false labels. The benchmark is designed to examine whether factual verification ability transfers from text to speech, and whether retrieval-augmented LALMs can use textual evidence to correctly support or refute spoken claims. Our experiments reveal a consistent text-speech modality gap: LALMs that verify written claims reliably often fail on the same claims when spoken. Moreover, retrieval alone provides limited gains because models frequently conflate retrieved evidence with the spoken claim. In contrast, retrieval combined with explicit reasoning improves claim-evidence comparison, with a thinking-tuned LALM reaching 86.1% accuracy. VeriSpeak highlights that effective speech misinformation detection requires not only speech understanding, but also grounded reasoning over retrieved evidence. The dataset is publicly available via Hugging Face at https://huggingface.co/datasets/abhiram4572/VeriSpeak.