Stanford 团队提出 Next-Concept Modeling，生成速度最高快 31 倍
Stanford 的 Hazy Research 组出了新生成框架 NCM，先写梗概再填细节，比扩散语言模型快最多 31 倍，推理任务还更强。
Hermann Kumbong、Ishan Khare 与 Stanford Hazy Research 团队提出 Next-Concept Modeling（NCM），先粗后细的并行文本生成框架。在相似参数量下，NCM 对比扩散语言模型（DLM），摘要和机器翻译等任务生成速度最高快 31 倍且质量相当或更好。无条件生成在 TinyStories、LM1B、OpenWebText 上快 10-21 倍，训练 token 用量少 3-5 倍。在 Sudoku 等推理任务上，NCM 以更少采样步数超过 AR 和扩散两类基线。
What if language models wrote the way that we think through ideas: rough sketch first with details last? New work by @KumbongHermann , @ishanskhare , and team introduces Next-Concept Modeling (NCM), a parallel generation framework achieving comparable or better quality than strong diffusion baselines while being 10-31× faster⚡️across unconditional generation, summarization & translation, and downstream reasoning tasks. Hermann Kumbong @KumbongHermann 🚀 Introducing Next-Concept Modeling (NCM): our new paradigm for fast, parallel, coarse-to-fine text generation. Compared to strong diffusion language models (DLMs) at similar parameter size, NCM achieves: ⚡ Up to 31× faster generation on downstream tasks like summarization and machine translation with comparable or better quality. 🏎️ 10–21× faster unconditional generation across TinyStories, LM1B, and OpenWebText, with comparable or better generative perplexity and using 3–5× fewer training tokens. 🧩 Better performance on reasoning tasks like Sudoku, outperforming both AR and diffusion baselines with fewer sampling steps. With @ishanskhare , @IddahMlauzi , @ekellbuch , @simran_s_arora , @Azaliamirh , and @HazyResearch . More details in the thread 🧵👇 Your browser does not support the video tag. 🔗 View on Twitter 🔗 View Quoted Tweet 💬 2 🔄 7 ❤️ 12 👀 1115 📊 5 ⚡
- arXiv cs.AI10-15 17:52原文
