对于在昇腾NPU上训练大规模MoE模型的团队，HyperParallel-MoE通过细粒度异构调度解决了通信与计算重叠不足的痛点，实测延迟降低1.58倍，值得关注并尝试集成到现有框架中。
本文提出了HyperParallel-MoE，一个针对昇腾NPU的MoE训练编译与调度框架。现有框架在昇腾NPU上串行执行MoE算子，未能充分利用其异构计算资源（矩阵AIC和向量AIV）。HyperParallel-MoE将算子级执行转化为静态调度的tile级异构任务流，通过AIV驱动单边通信消除主机侧同步，并实现通信与计算在单个内核启动内的细粒度重叠。在MindSpore和MindFormers栈中实现，基于DeepSeek风格MoE模型在昇腾A3集群上测试，将Dispatch-to-Combine的MoE-FFN延迟最多降低1.58倍。这项工作表明tile级异构调度能显著提升现代NPU上的MoE训练效率。
HyperParallel-MoE: Multi-Core Interleaved Scheduling for Fast MoE Training on Ascend NPUs
Modern Mixture-of-Experts (MoE) models increasingly rely on large-scale AI accelerator clusters for efficient training. Ascend NPUs expose heterogeneous on-chip compute resources, including matrix-oriented AIC units and vector-oriented AIV units with explicit cross-queue synchronization support. However, existing training frameworks largely execute MoE operators in a serialized kernel-by-kernel manner, leaving substantial heterogeneous parallelism underutilized. This paper presents HyperParallel-MoE, a compilation and scheduling framework for MoE training on Ascend NPUs. HyperParallel-MoE transforms operator-level MoE execution into a statically scheduled tile-level heterogeneous taskflow spanning AIC and AIV resources. It introduces AIV-driven one-sided communication to eliminate host-side collective synchronization, dependency-preserving tile task generation to unify communication and computation under a common task abstraction, and event-driven static scheduling to coordinate cross-queue execution with low runtime overhead. HyperParallel-MoE further executes the compiled taskflow within a unified runtime that concurrently drives AIC and AIV workers inside a single kernel launch, enabling fine-grained overlap among communication, matrix computation, and vector computation while preserving existing optimized operators. We implement HyperParallel-MoE in the MindSpore and MindFormers stack and evaluate it using DeepSeek-style MoE models on Ascend A3 clusters. Across multiple expert-parallel configurations, HyperParallel-MoE reduces Dispatch-to-Combine MoE-FFN latency by up to 1.58x, demonstrating that tile-level heterogeneous scheduling can substantially improve MoE training efficiency on modern NPUs.