论文精选

离散贝克曼传输模型用于一步语言建模和推理

Discrete Beckmann Transport Models for One-Step Language Modeling and Reasoning

精选理由

这是篇挺有意思的论文,作者提出了一种新模型,能一步生成文本,比传统方法更快,而且质量还更好。

这篇论文提出了一种名为离散贝克曼传输模型(DBTM)的新方法,它基于时间无关的流,能在一步内将任何点映射到简单形的顶点。该方法通过最小化数据中的守恒方程残差,移除了对教师模型和时序条件的需求。在语言建模和推理任务上,DBTM实现的一步和少步生成在质量和准确性上优于离散扩散和连续流基线。

原文 · arXiv cs.LG

Discrete Beckmann Transport Models for One-Step Language Modeling and Reasoning

Discrete diffusion and flow models are a promising alternative to autoregressive language models, but compressing many-step sampling into fewer steps typically requires distilling a pretrained teacher model. This caps the student at the teacher's quality and requires a costly two-stage training pipeline. We introduce Discrete Beckmann Transport Models (DBTM), built on a time-independent flow whose autonomous transport map provably carries any point in the ambient space to a fixed point on the vertices of the simplex in a single step. We show that this fixed-point property is characterized by a conservation equation whose residual can be minimized directly from data, removing the requirement for a teacher flow and time conditioning. Under this construction, a partially trained map corresponds to the flow truncated at finite time, so generation reduces to iterating one map until it reaches a fixed point. We further extend the map to a partial-context interpolant where additional function evaluations act as refinement steps rather than ODE integration steps. On language modeling and reasoning tasks, DBTM enables one- and few-step generation that improves quality and accuracy over discrete diffusion and continuous flow baselines.