Stanford 提出 Halcyon：无需语言预训练的文生图模型
Text-to-Image Generative Modeling Without Language Pretraining
Stanford 训了一族不靠语言预训练的文生图模型 Halcyon，算力省几百倍还能打赢 LlamaGen，思路挺反直觉，代码也放出来了。
Stanford 团队发布 Halcyon，一族从头预训练的早融合多模态自回归模型，训练数据为 100M 图文对的 GPIC 数据集，完全不依赖互联网规模的语言预训练。Halcyon-0.52B 在 gFD 指标上以 48.0 优于 LlamaGen-XL/8 的 63.3，预训练算力少 18.2 倍；Halcyon-1.6B 以 35.3 gFD 超过 JiT-T2I-XXL/8++ 的 41.0，算力少 275 倍。受控实验显示纯文本预训练并不能提升文生图效果。通过把 caption dropout 替换为 I2T 监督，Halcyon 还能在同等预算下学会图像描述生成。代码和模型已开源。
Text-to-Image Generative Modeling Without Language Pretraining
Modern text-to-image (T2I) systems use internet-scale language pretraining for text-conditioned image generation. We ask whether language pretraining is necessary for strong T2I generation. We introduce Halcyon, a family of early-fusion multimodal autoregressive models pretrained entirely from scratch on GPIC, a dataset of 100M image-text pairs. Halcyon-0.52B outperforms LlamaGen-XL/8 (48.0 vs. 63.3 gFD) with 18.2$\times$ less pretraining compute. Further, Halcyon-1.6B outperforms JiT-T2I-XXL/8++ (35.3 vs. 41.0 gFD) with 275$\times$ less pretraining compute. Controlled experiments show that text-only pretraining does not improve T2I generation. Halcyon also learns image captioning by replacing caption dropout with I2T supervision, maintaining competitive T2I generation quality under the same pretraining budget. We show that strong T2I systems can be trained from scratch without internet-scale language pretraining. Code and models are available at https://cs.stanford.edu/~keshik/halcyon
