TorchTitan 更新：支持 DeepSeek-V4 等 5D 并行，新增 TitanRL 框架
PyTorch 官方的 TorchTitan 大更新，DeepSeek-V4 和 Qwen3.5 都能跑 5D 并行了，Muon 优化器还能提速 79%，训大模型的得看看。
TorchTitan 在 PyTorch 原生框架中加入完整 5D 并行支持，覆盖 DeepSeek-V4、Qwen3.5、Kimi K3 和 Muse Spark Open 等模型。新发布的 TitanRL 框架统一了后训练与智能体工作负载的流程。FlexShard 可将分布式 Muon 优化器耗时最多降低 79%，torch_checkpointing 在 512 张 GPU 上将 checkpoint 阻塞时间压缩到 34 毫秒。
TorchTitan has released major performance and architecture updates in native PyTorch, adding full 5D parallelism support for DeepSeek-V4, Qwen3.5, Kimi K3, and Muse Spark Open alongside the new TitanRL framework for unified post-training and agentic workloads.
Scaling efficiency improves further with FlexShard, which slashes distributed Muon optimizer time by up to 79 percent, and torch_checkpointing, which drops checkpoint blocking times down to 34 milliseconds on 512 GPUs.
Read our latest blog to learn more about the latest improvements and how to get started: https://t.co/y8NaXrOyuD