H-CDLM：分层联合扩散框架提升连续扩散语言模型
Denoising Hierarchical Representations: Joint Continuous Diffusion for Language Modeling
arXiv 上这篇论文教扩散语言模型同时建模 token 和粗粒度聚类表示，H-CoBit 在 LM1B 上 GenPPL 比基线低 24.2 分，代码也放出来了。
论文提出 Hierarchical Continuous Diffusion Language Models（H-CDLM），对 token 及其嵌入聚类得到的粗粒度表示同时做扩散，并允许每个模态使用独立的采样器与调度。应用到 CoBit 得到的 H-CoBit 在 LM1B 上生成困惑度（GenPPL）达 49.4、OWT 上达 50.4，比基线分别降低 24.2 和 20.7 分，超过同规模离散扩散语言模型。GSM8K 上准确率为 27.4%，高于此前连续扩散与流匹配模型。同一框架套用到流匹配模型 FLM 得到 H-FLM，同样带来一致提升。
Denoising Hierarchical Representations: Joint Continuous Diffusion for Language Modeling
Diffusion Language Models (DLMs) hold the promise of order-agnostic, parallel text generation. Recently, continuous diffusion and flow matching models have seen substantial gains, driven by carefully crafted token representations and diffusion/flow spaces. In this work, we introduce Hierarchical Continuous Diffusion Language Models (H-CDLMs), a simple framework that further improves continuous DLMs with minimal compute and parameter overhead. Drawing on the discrete DLM and continuous image diffusion literature on joint diffusion, we diffuse multiple modalities in parallel. These modalities represent tokens at different semantic granularities: in our instantiation, the tokens themselves and coarser clusters obtained by clustering pretrained token embeddings. We propose a general setup that allows per-modality samplers and schedules to enhance the interplay between modalities. Applied to CoBit, this yields H-CoBit, which delivers large empirical gains across benchmarks. At dataset entropy, H-CoBit improves MAUVE and reaches a generative perplexity (GenPPL) of 49.4 on LM1B and 50.4 on OWT, improving on the baseline by 24.2 and 20.7 points and surpassing even discrete DLMs of comparable size. On GSM8K, it reaches 27.4% accuracy, outperforming prior continuous diffusion and flow-based models. We further apply H-CDLM to the flow matching model FLM, obtaining consistent gains with H-FLM and demonstrating that the framework generalizes across continuous generative paradigms. Our code will be made publicly available at https://github.com/matol-16/HCDLM.git .