产品多源确认76°

PyTorch推出原生TPU后端引擎vLLM和SGLang

精选理由

Google和Meta工程师将分享如何让vLLM和SGLang在TPU上原生运行,保留GPU用户熟悉的API和工作流。

Google和Meta工程师将在PyTorch Conference North America介绍基于TorchTPU的新vLLM和SGLang引擎。

正文

这些引擎使现有基础设施能在TPU上原生运行,保留GPU用户熟悉的调度器、批处理系统和OpenAI兼容API。新模型和功能可在TPU上运行,无需单独的TPU实现,生产环境运行经验包括编译时间、设备放置和KV缓存布局。

原文 · PyTorch

Want to hear more about new vLLM and SGLang engines built on a new PyTorch-native TPU backend, TorchTPU?

Find Qi Zhou of @Google, Colin Taylor and Angela Yi of @Meta at PyTorch Conference North America in San Jose where they will discuss how they focused on making the existing @sgl_project & @vllm_project infrastructure work natively on TPU, preserving the schedulers, batching systems, OpenAI-compatible APIs, and torch.compile workflows already familiar to GPU users.

Because the serving engines remain upstream, new models and features can run on TPU without requiring separate TPU-specific implementations. Learn more about how it works across the stacks & what they learned running this in production, including compile time, device placement under tracing, KV cache layout, and cross-chip collectives.

Register for #PyTorchCon NA now: https://t.co/jBApW8ocHQ

产业坐标与信源核验

本条情报归属 AITOP「产品」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:vLLM、SGLang、PyTorch、TPU,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「PyTorch」(@PyTorch),经 AITOP 多维评分管线结构化提炼。截至目前,全网已有 11 家独立信源对该事件进行交叉印证与跟进报道(含 向阳乔木、Rappler: Technology、Xianbao Qian 等),属于近期高关注度行业信号。