解读 Qwen3-TTS 声音克隆模型内部表示的可解释性研究
Discovering Interpretable Acoustic Features in Autoregressive Voice-Cloning TTS Models
用稀疏自编码器拆解 Qwen3-TTS 到底记住了什么，684 个特征还能自动验证，做 TTS 可解释性的可以看看。
一篇 arXiv 论文针对 Qwen3-TTS 等自回归声音克隆 TTS 模型提出可解释性框架，用稀疏自编码器（SAE）、PCA 和随机方向做稀疏探测、 steering 和自动可解释性分析。研究扩展到 6 个模型，发现这些模型编码了音高、能量、噪声、口音等概念，但探测有效不代表 steering 有效。SAE 适合稀疏探测和新概念发现，PCA 在密集探测和 steering 上表现相当。自动可解释性在 Qwen3-TTS 中验证了 684 个 SAE 特征，涵盖说话人聚类、说话风格和录音条件。
Discovering Interpretable Acoustic Features in Autoregressive Voice-Cloning TTS Models
Voice-cloning TTS models synthesize an unseen speaker's voice from brief speech prompts, but it remains unclear which acoustic properties they represent and use. We introduce a feature-based interpretability framework for autoregressive voice-cloning TTS models, analyzing features from sparse autoencoders (SAEs), PCA, and random directions via sparse probing, steering, and quantitative auto-interpretability. We focus on Qwen3-TTS and extend probing and steering to $5$ additional models to test generalization across architectures. These models encode concepts like pitch, energy, noise, and accent, but sparse prediction and steering are effective only for some concepts. Strong probing does not always translate to steering or vice-versa. SAEs are useful for sparse probing of human-labelled concepts and novel-concept discovery, while PCA is competitive for dense probing and steering. Auto-interpretability finds $684$ automatically validated SAE features in Qwen3-TTS covering speaker clusters, speaking style, and recording conditions beyond known concepts. Overall, we perform a comprehensive interpretability study of autoregressive voice-cloning TTS models.
