论文精选

提出新方法让语音大模型直接识别情绪,提升准确率并减少错误

Reading Emotions in the Token Space: Discriminative Adaptation of SpeechLLMs for Emotion Recognition

精选理由

这个研究方法挺有意思,它让语音大模型直接读出情绪标签,而不是生成,这样应该能减少错误,提升准确率。

研究人员提出一种新的方法,让语音大模型(SpeechLLMs)通过分类头直接读取最终提示词的隐藏状态来识别情绪,而不是通过生成器。这种方法在IEMOCAP数据集上测试,在两个语音大模型架构上提升了宏F1分数,并减少了幻觉,尤其是在处理真实ASR转录文本时效果更明显。

原文 · arXiv cs.AI

Reading Emotions in the Token Space: Discriminative Adaptation of SpeechLLMs for Emotion Recognition

SpeechLLMs have shown strong potential for emotion recognition, yet they read the predicted emotion off a generative decoder not suited for classification: it can emit labels outside the target set and favors frequent classes. We propose a discriminative adaptation that reads the final prompt token's hidden state through a classification head, producing a label in one forward pass without modifying the backbone. Because this readout starts from the hidden state the model would otherwise decode, it gives a controlled comparison of generative and discriminative inference in an otherwise identical speechLLM. We keep the head a single linear layer, trading little accuracy for interpretability: each emotion becomes one direction in the LLM output token space, revealing associated tokens. On IEMOCAP, across two speechLLM architectures, it improves Macro F1 and removes hallucinations, with largest gains on realistic ASR transcripts. Our analysis reveals that these emotion directions encode indirect associations mirroring biases in web-scale text.