Primus Projection：单机仿真加自动调优，替代千卡分布式训练试错
Beyond the Trial-and-Error Loop: Hybrid Projection and Automated Tuning for Distributed Training
调千卡训练配置不用再烧 GPU 试错了，单卡跑不到 30 分钟，就给 Mixtral 8x22B 找到比官方参考快 27% 的配置。
分布式 LLM 训练要在张量、流水线、专家、上下文、数据并行与微批大小等维度联合调参，团队通常靠多节点反复试错烧掉大量 GPU 时。Primus Projection 用混合投影引擎解决：单卡 GPU 微基准测量计算内核，再通过降规模-测量-升规模流程在 CPU 上还原多节点通信、FSDP overlap 与完整流水线调度，支持完全无 GPU 的纯仿真模式。投影精度方面，Llama 3.1 与 Mixtral 8x22B 的误差保持在实测多节点吞吐的 ~10% 以内，DeepSeek V3 671B 在 1,024 卡上误差 4.7%，Qwen3-235B-A22B 在 256 卡上误差 1.9%。Primus Tuning Agent 把投影引擎当作秒级打分器，用确定性种子规划器加 LLM 探索循环搜索跨维度组合，并以解析内存预过滤剔除不可行候选。在 Mixtral 8x22B 案例中，该智能体用不到 30 分钟的单节点探索，找到比公开 4 节点 BF16 参考配置吞吐高 27% 的方案。
Beyond the Trial-and-Error Loop: Hybrid Projection and Automated Tuning for Distributed Training
Configuring distributed LLM training across hundreds of GPUs requires jointly optimizing tensor, pipeline, expert, context, and data parallelism alongside micro-batch size, recomputation, sharding, and overlap, a combinatorial space where memory legality can hinge on a single byte of activation accounting. Today, teams navigate it through expensive multi-node trial and error, burning GPU-hours on crashes and suboptimal recipes. We present Primus Projection, which closes this loop with two coupled components. First, a hybrid projection engine that measures what it can and simulates what it cannot: GPU microbenchmarks on as few as a single device capture compute kernels and any in-scope communication; a sub-node downscale-measure-upscale workflow analytically restores out-of-scope collectives, FSDP overlap, and the full pipeline schedule at the multi-node target. A pure simulation mode runs entirely on CPU, enabling pre-silicon planning without any GPU. Second, the Primus Tuning Agent treats the projection engine as a fast scoring oracle (seconds per candidate instead of tens of minutes) and uses it to drive an intelligent search over the configuration space. A deterministic seed planner sweeps single axes; an LLM investigation loop proposes cross-axis combinations the planner cannot reach, with an analytic memory pre-filter that rejects infeasible candidates before any GPU time is spent. Validated on Llama 3.1 (dense) and Mixtral 8x22B (MoE) across two GPU generations, all projections stay within ~10% of measured multi-node throughput, and fidelity holds at production scale: 4.7% error on DeepSeek V3 671B at 1,024 GPUs and 1.9% on Qwen3-235B-A22B at 256 GPUs. In a Mixtral 8x22B case study, the Primus Tuning Agent discovers a configuration delivering +27% throughput over the published 4-node BF16 reference at the same scale, in under 30 minutes of single-node exploration.
