ToR-OT:用自回归 token 回归学习最优传输方案
Learning Optimal Transport Plans Via Autoregressive Token Regression
把最优传输当成序列生成来解,预训练一次就能零样本处理新分布和异构成本,推理时不用跑优化器。虽然还没法替代经典求解器,但如果你研究摊销优化或神经求解器,这篇的稀疏解码思路值得借鉴。
论文提出 ToR-OT,一个基于 transformer 的模型,把最优传输(OT)问题重构为结构化序列生成任务,通过自回归 token 回归预测离散 OT 传输方案。作者在大量合成 OT 实例上预训练,单个模型即可 zero-shot 泛化到未见过的分布对和异构成本度量,无需重新训练。
技术上的核心是利用精确离散 OT 解天然具有的稀疏性:解码器只预测非零条目,输出稀疏原始解,避免了稠密耦合矩阵的生成。推理阶段不再运行优化求解器,而是直接解码序列,实现摊销式推断;在低维离散问题上,ToR-OT 给出准确的近似方案,并优于按实例训练的神经 OT 基线。
作者明确指出 ToR-OT 尚不能替代经典求解器,经典 solvers 仍是该任务的金标准。论文的价值在于提供了一个有真值解、边际约束显式、稀疏性可刻画的测试床,用以研究自回归 transformer 能否内化此类约束优化问题的结构。
Learning Optimal Transport Plans Via Autoregressive Token Regression
Optimal Transport (OT) provides a principled framework for finding the minimum cost coupling between probability distributions. Classical solvers remain the gold standard for this task. At the same time, OT is an ideal testbed for studying whether learning-based amortized models can internalize the structure of OT type problems. Here, ground-truth solutions are available and marginal constraints are explicitly defined. Also, exact discrete solutions have well-characterized sparsity. We describe ToR-OT, a transformer-based model that learns to predict discrete OT plans through autoregressive token regression. We reframe OT into a structured sequence generation task. ToR-OT decodes sparse transport plans as sequences instead of solving an optimization task at test time. We focus on three properties: (a) Zero-shot generalization: pretraining on a large synthetic corpus of OT instances allows a single model to generalize to unseen distribution pairs and heterogeneous cost metrics without retraining, (b) Sparse primal decoding: by exploiting the inherent sparsity in exact discrete OT solutions, the decoder predicts only nonzero entries (avoids dense coupling outputs), (c) Amortized inference: on low-dimensional discrete problems, ToR-OT gives accurate approximate plans and compares favorably with per-instance neural OT baselines. Overall, ToR-OT is not yet a replacement for classical solvers, but a step toward understanding how autoregressive transformers can learn structure of constrained optimization problems like OT.
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:Optimal Transport、ToR-OT、transformer、摊销推断,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Ivan Jaen Marquez, Sotirios Panagiotis Chytas, Vikas Singh),经 AITOP 多维评分管线结构化提炼。