MMAS 开源模型实现逐 0.5 秒音乐情绪连续估计
Concordance-Optimal Continuous Music Emotion Estimation: Variance Decomposition, Dual-Branch Temporal Fusion, and Conformal Prediction Intervals
有人开源了能逐 0.5 秒追踪歌内情绪变化的 MMAS，还顺手指出业界指标其实测不出歌内动态，做音乐分析的研究者可以看看。
论文提出 MMAS（Multimodal Music Analysis System），从原始音频每 0.5 秒预测 Arousal 和 Valence。作者将标注方差分解为歌曲间与歌曲内两部分，指出常用的池化帧级相关系数主要被歌曲间差异主导，并引入窗口化 Lin's CCC 来聚焦歌曲内动态。在 DEAM 的 1,764 首歌上调节权重 λ 从 0 到 1，池化 Arousal r 从 0.843 降到 0.810，但歌曲内平均 r 从 0.204 提升到 0.368。MMAS 融合 MERT-v1-95M 嵌入与十个符号特征，在 DEAM 和 PMEmo 共 2,531 首歌上联合训练，池化 r 达 0.801/0.685（DEAM）和 0.742/0.746（PMEmo），歌曲内 Arousal r 相比 ridge 基线约三倍（0.344 vs 0.125）。模型以 pip 包形式开源，split-conformal 区间在 95% 目标下达到 93.7%（Arousal）和 90.0%（Valence）覆盖率。
Concordance-Optimal Continuous Music Emotion Estimation: Variance Decomposition, Dual-Branch Temporal Fusion, and Conformal Prediction Intervals
Continuous music emotion recognition (MER) estimates how Arousal and Valence evolve within a song, yet the field's standard metric, pooled frame-level correlation, is dominated by between-song differences and cannot distinguish a model that tracks emotional dynamics from one that only ranks songs by their average level. We decompose annotation variance into between- and within-song components and show that mean squared error (MSE) targets the former, while windowed Lin's concordance correlation coefficient (CCC = $r \cdot C_b$) targets the latter by penalising the variance compression MSE induces. A single weight $λ$ controls the trade-off: on 1,764 DEAM songs, raising $λ$ from 0 to 1 lowers pooled Arousal $r$ from 0.843 to 0.810 while raising mean within-song $r$ from 0.204 to 0.368. Building on this objective, we present MMAS (Multimodal Music Analysis System), an open-source model that predicts Arousal and Valence every 0.5 s from raw audio. It fuses frozen MERT-v1-95M embeddings with ten symbolic features from automatic transcription through a dual-branch temporal convolutional network with cross-modal attention. Because both branches are audio-derived, one model trains jointly on DEAM and PMEmo without domain conditioning. Under song-grouped 5-fold cross-validation (2,531 songs), pooled $r$ reaches 0.801/0.685 (Arousal/Valence) on DEAM and 0.742/0.746 on PMEmo. Against a ridge baseline on identical features, within-song Arousal $r$ roughly triples (0.344 vs. 0.125; $n = 353$ songs, Wilcoxon $p = 1.7 \times 10^{-21}$, $d = 0.56$). Complete transcription failure costs at most 0.04 pooled $r$. Split-conformal intervals attain 93.7% (Arousal) and 90.0% (Valence) marginal coverage at a 95% target, with the shortfall traced to within-song dependence. MMAS is released as a pip-installable package.