Apple 提出三模态掩码扩散模型 MDM，系统分析设计空间
The Design Space of Tri-Modal Masked Diffusion Models
扩散模型也能同时搞定文字、图像和音频？Apple 这篇论文把多模态训练的配方拆得很细。
Apple 研究团队发布了三模态 Masked Diffusion Models（MDM）的研究，首次在文本、图文、音频-文本三类数据上从头预训练离散扩散模型。这与此前先微调单模态基座再做双模态生成的路线不同，MDM 直接统一处理三种模态组合。论文系统分析了多模态 scaling laws、模态混合比例、噪声调度和 batch size 对训练的影响，并给出推理采样的默认配置。在 batch size 分析中，作者还提出了一种基于随机微分方程（SDE）的新方法。
The Design Space of Tri-Modal Masked Diffusion Models
Discrete diffusion models have emerged as strong alternatives to autoregressive language models, with recent work initializing and finetuning a base unimodal model for bi-modal generation. Diverging from previous approaches, we introduce the first tri-modal Masked Diffusion Models (MDM) pretrained from scratch on text, image-text, and audio-text data. We systematically analyze multimodal scaling laws, modality mixing ratios, noise schedules, batch-size effects and provide optimized inference sampling defaults. Our batch-size analysis yields a novel stochastic differential equation (SDE) based…
