AMD 在 PyTorchCon 展示 LLaDA-8B 扩散语言模型 NPU 端侧推理
AMD 把扩散语言模型 LLaDA-8B 跑进了 NPU，比 CPU INT4 快 1.67 倍、省电 8.9 倍，做端侧推理的可以看看他们的量化与图切分细节。
AMD 团队将在 PyTorch Conference North America 2026 展示在 XDNA2 NPU 上部署 LLaDA-8B-Instruct 的方案。在 Ryzen AI MAX+ 395、序列长度 128 的条件下，NPU 路径每次前向耗时 504 ms、功耗 36 W，而 CPU INT4 为 843 ms 和 110 W，速度提升 1.67 倍，能效提升 8.9 倍。海报还涵盖 INT4 量化、图切分流程、CastAvx 修复和单步 prefill 的运行时实现。
Ravi Gupta, Shiksha Patel, and Rishi Madduri (@AMD) will present “Efficient Inference of Masked Diffusion Language Models on AMD XDNA2 NPU” at #PyTorchCon North America 2026.
Their poster presents a power-efficient deployment of LLaDA-8B-Instruct on an AMD NPU. On a Ryzen AI MAX+ 395 APU at sequence length 128, the NPU vendor path ran at 504 ms per forward at 36 W, compared with 843 ms and 110 W for CPU INT4, making it 1.67x faster and 8.9x better on energy.
The poster will also cover the INT4 quantization and graph-partitioning pipeline, the CastAvx fix, and the runtime approach used for single-step prefill on the XDNA2 NPU.
Register for PyTorch Conference North America 2026: https://t.co/jBApW8nESi