PyTorch 2.15 发布,NCCL 新后端成为默认
PyTorch 2.15 更新了:NCCL 新后端转正,FSDP2 内存控制更细,Blackwell 上 NVFP4 解码提速。做训练推理的可以看看直播问答。
PyTorch 2.15 将重写的 NCCL 后端设为默认,新增基于 NIXL 的单边传输通信方式。FSDP2 和流水线并行获得更细粒度的内存与通信控制。解码速度方面,NVIDIA Blackwell GPU 上的 4-bit(NVFP4)LLM 解码更快,Apple Silicon 上新增 FP8 支持并加快解码。团队将于 11 月 5 日上午 11 点(太平洋时间)举行直播问答。
Bring your questions about PyTorch 2.15 to our live Q&A TOMORROW, November 5, at 11 AM PT.
Tristan Rice (@rice_fry), Natalia Gimelshein, Huy Do, and Jane Xu (@janeyx99) (@Meta), with moderator Chris Gottbrath (Gottbrath Tech), will share an overview of the release and answer community questions about PyTorch and the new capabilities in the release.
PyTorch 2.15 makes the rewritten NCCL backend the default, adds one-sided transports with a NIXL backend, and gives FSDP2 and pipelining finer memory and communication control. The release also brings faster 4-bit (NVFP4) LLM decoding on NVIDIA Blackwell GPUs, FP8 and faster decode on Apple Silicon, and broader platform support.
Register: https://t.co/opFmSkAUNb