Together AI 发布 Together Compiler：面向 AI 推理的持续编译系统
Together Compiler: Continual Compilation for AI Inference
Together AI 把推理编译交给 Agent 循环自动调优，Llama 1B 上比手写 Megakernels 快 1.7 倍，InferenceX 上比开源引擎高 2.6 倍 TPS。做推理部署和性能优化的工程师值得看看这篇论文的机制细节和生产经验。
Together AI 发布 Together Compiler（TKCC），定位为首个专为 AI 推理优化设计的持续 Agent 编译循环，论文已挂 arXiv。系统针对现有方案的吞吐量-保真度权衡，提出 Tiered Persistent Engine 将引擎启动开销与前向计算解耦，并用成本模型选择并行配置来初始化优化过程。
TKCC 在 Llama 1B 工作负载上比高度优化的手写 Megakernels 快至 1.7 倍，在 InferenceX 基准上比最好的开源引擎在 Kimi K3、GLM 5.3、MiniMax M3.1 Flash、DeepSeek v4.1 Flash 等前沿模型上最高提升 2.6 倍 TPS。batch size 1 下它创下了新纪录：M3.1 Flash 达 1111 TPS，DeepSeek v4.1 Flash 达 1205 TPS。
论文还包含对 Agent 上下文管理的消融实验和生产部署经验，目前 TKCC 已在生产集群中服务数万亿 token。该系统通过 Agent 爬山法持续应对模型发布频繁、架构多样化、流量分布多变的优化难题，将编译循环变成自动化流程。
Together Compiler: Continual Compilation for AI Inference
Today's AI workloads are rapidly changing: new models are released more often, model architectures are increasingly diverse, and traffic distributions change frequently. The challenge we face at Together AI is continual optimization for these changing workloads. Compiling a deployment into an optimized forward pass is a challenge that is well-suited for agent hill-climbing. Existing systems struggle with continual compilation due to a tradeoff between measurement throughput and fidelity. Many systems use agents to optimize individual kernels from the inference forward pass using offline benchmarks, but timings are low-fidelity because of data-dependent behavior in primitives like mixture-of-experts (MoE). On the other hand, systems use agents to directly optimize end-to-end engines but suffer from slow engine launch times. To address these challenges, we build Together Compiler (TKCC), the first truly continual agent loop designed for AI inference optimization. To address the throughput-fidelity tradeoff, we propose: (1) a Tiered Persistent Engine that decouples engine launch overheads from forward pass computation; (2) a cost model to select the parallelism configurations that seed the optimization process. We also conduct extensive ablations on agent context management and present our insights on deploying TKCC in production. TKCC outperforms highly-optimized hand-written Megakernels on the Llama 1B workhorse workload by up to 1.7x and the best open source engines on the latest frontier models (Kimi K3, GLM 5.3, MiniMax M3.1 Flash, DeepSeek v4.1 Flash) across hardware platforms by up to 2.6x higher TPS on the InferenceX benchmark. TKCC also sets the state-of-the-art performance at batch size 1, with 1111 TPS on M3.1 Flash and 1205 TPS on DeepSeek v4.1 Flash. Finally, TKCC generalizes beyond frontier language model inference. TKCC serves trillions of tokens across our production fleet.
本条情报归属 AITOP「论文」核心栏目，重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」「中美大模型技术代差与商业化对照」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：Together AI、推理优化、编译器、DeepSeek v4.1 Flash，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始信息发布自一手权威信源「arXiv: DeepSeek」（Tier-1 一手发布），具备高确定性参考价值。截至目前，全网已有 7 家独立信源对该事件进行交叉印证与跟进报道（含 Together AI、Lxfater、berryxia 等），属于近期高关注度行业信号。
