这篇论文解释了为什么音频编解码器在6.25Hz会突然变差,原来不是理论限制而是训练设置问题,修正后能降到1.6Hz依然可用。
这篇论文通过控制帧率消融实验,发现神经音频编解码器在6.25 Hz处存在质量悬崖,并排除了音素冲突和码本饱和两种假设。作者指出问题源于固定训练片段时长导致解码器缺乏帧间上下文,而修正后词错误率(WER)在3.1 Hz和1.6 Hz下仍随音素负载平滑下降。研究结果表明,低帧率编解码器的推理效率增益比此前认为的更易实现。
Probing Low Frame Rate Degradation in Neural Audio Codecs
Low frame rates in neural audio codecs are attractive for autoregressive speech synthesis, where the generation cost scales linearly with the sequence length. Recent work has demonstrated that codecs can operate at 12.5 Hz and below, but the mechanisms underlying low frame rate degradation remain insufficiently understood. We investigate these mechanisms through a controlled frame rate ablation. We reproduce a quality cliff at 6.25 Hz reported in previous works and evaluate candidate explanations: phonemic collisions and codebook saturation, neither of which shows evidence of a fundamental barrier. The cliff is instead caused by suboptimal training configuration: fixed clip duration during training yields too few tokens at low frame rates, starving the decoder of inter-token context. Once corrected, WER degrades smoothly with phonemic load down to 3.1 Hz and 1.6 Hz, suggesting the inference-time efficiency gains of low frame rate codecs are more accessible than previously assumed.