MERP：面向缺失模态的多模态多标签情绪识别框架
MERP: A Latent-Bottleneck Framework for Robust Multi-Modal Multi-Label Emotion Recognition
这个 MERP 框架专门解决测试时部分模态缺失就崩的问题，只喂音频也能跑出 0.473 的 F1，代码还开源了。
MERP 是一个多模态多标签情绪识别框架，将联合序列映射到固定大小的潜变量库，使推理不依赖原始输入长度。它在 20% 测试时随机模态缺失下，micro-F1 在 CMU-MOSEI 达 0.563、M3ED 达 0.780，超过最强基线 0.004 和 0.005。在 CMU-MOSEI 仅音频的严苛场景中保留 0.473 micro-F1，而基线 LDDU 仅为 0.022。训练时不需缺失模态样本，代码已在 GitHub 开源。
MERP: A Latent-Bottleneck Framework for Robust Multi-Modal Multi-Label Emotion Recognition
Multimodal multi-label emotion recognition is important for affect-aware systems, but models trained with complete text, visual, and acoustic streams often degrade sharply when some streams are unavailable at deployment. We address this train-test gap by learning a compact representation that preserves emotion-specific evidence from any available subset of modalities. We propose the Multi-modal Emotion Recognition Perceiver (MERP), which maps the joint sequence into a fixed-size latent bank, thereby keeping repeated reasoning independent of raw input length; it extracts label-specific evidence before and after latent reasoning and aligns the two levels through reconstruction. MERP further uses positive and negative class prototypes, a noise-gated feature enhancer with frequency-shaped perturbations during training, and confidence-aware class-wise fusion to strengthen separability, reduce sensitivity to missing inputs, and combine complementary predictors. Under 20 percent test-time random modality drop, MERP achieves micro-F1 scores of 0.563 on CMU-MOSEI and 0.780 on M3ED, exceeding the strongest evaluated baselines by 0.004 and 0.005, respectively; in the severe CMU-MOSEI audio-only setting, it retains 0.473 micro-F1 versus 0.022 for LDDU. These results show that fixed latent reasoning provides a practical, testable route to robust multimodal emotion recognition without requiring missing-modality examples during training. Code is available at https://github.com/Creative-zcx/MERP.