研究:人声合唱多音高估计中线性STFT效果超过HCQT
Revisiting Input Time-frequency Representations in Multi-pitch Estimation for Vocal Ensembles
做音乐信息检索或人声分离的朋友可以看看,这篇实测发现简单便宜的 STFT 竟比 HCQT 效果更好,训练特征提取成本还低不少。
arXiv 论文比较了人声合唱多音高估计(MPE)中常用的谐波常数 Q 变换(HCQT)与线性短时傅里叶变换(STFT)两种输入表示。实验显示,线性 STFT 虽然频率分辨率固定、缺少音高对齐的输入网格,效果反而超过 HCQT,并大幅降低特征提取成本。分析还发现更长的分析窗口或更宽的频谱覆盖没有额外收益,而将输入限制在预测音高范围内反而削弱线性 STFT 的优势。
Revisiting Input Time-frequency Representations in Multi-pitch Estimation for Vocal Ensembles
Multi-pitch estimation in vocal ensembles is challenging because singers occupy overlapping pitch ranges and often sing at closely spaced fundamental frequencies, causing their harmonics to overlap in time-frequency representations. Existing models commonly use harmonic constant-Q transform (HCQT)-based representations to provide frequency-adaptive resolution, at the cost of expensive feature extraction when training mixtures are generated on the fly. We revisit this design and compare HCQT with a linear short-time Fourier transform (STFT), whose frequency bins are directly provided as model inputs. Despite its fixed frequency resolution and the absence of a pitch-aligned input grid, the linear STFT outperforms HCQT while substantially reducing feature-extraction cost. Further analysis shows that a longer analysis window or broader spectral coverage provides no additional improvement, while restricting the input to the predicted pitch range reduces the advantage of the linear STFT. These results suggest that finer frequency resolution does not necessarily improve vocal-ensemble MPE, and that shorter analysis windows can be more effective for time-varying vocal pitches.