vLLM Day-0 支持 Baidu ERNIE 5.5 Flash Base 开源权重
Baidu 开源了 ERNIE 5.5 Flash Base，vLLM 第一天就能跑，代码基准比 DeepSeek-V4-Flash-Base 高一截，部署配方也给了。
Baidu 发布 ERNIE 5.5 Flash Base 开源基础模型，总参数 280B、每 token 激活 14B，预训练数据 33.5T tokens，训练上下文 128K，微调和推理可扩展至 1M tokens。vLLM 提供 Day-0 支持，代码评测上 HumanEval+ 达 82.3%、LiveCodeBench 达 40.1%（均 1-shot），同设置下高于 DeepSeek-V4-Flash-Base 的 69.5% 和 31.9%。架构上采用 37 层压缩注意力加 6 层 MLA-DSA、Multi-Router Latent MoE，以及用最后一层做投机草稿的 ECHO 机制。权重已在 Hugging Face 和 ModelScope 上线，FP8 与 BF16 部署配方在 vLLM 0.32.0 上经 NVIDIA GB200 和 H100 验证。
🎉 Day-0 support for @Baidu_Inc’s ERNIE 5.5 Flash Base with vLLM!
280B total parameters, 14B active per token. Pre-trained on 33.5T tokens, with 128K training context and up to 1M tokens for fine-tuning and inference. Open Base weights for domain adaptation, instruction tuning and RL research.
Code highlights: 82.3% HumanEval+ and 40.1% LiveCodeBench pass@1 (both 1-shot), versus 69.5% and 31.9% for DeepSeek-V4-Flash-Base under the same evaluation setup.
Efficiency is built into the architecture: 37 compressed-attention layers + 6 MLA-DSA layers, Multi-Router Latent MoE, and ECHO — reusing the final layer as a speculative draft.
Serve FP8 or BF16 with Baidu’s vLLM-ERNIE plugin on vLLM 0.32.0. ECHO runs through vLLM’s MTP path; recipes include expert parallelism, KV offload and prefill/decode disaggregation, with runs verified on @NVIDIA GB200 and H100.
Weights are on @huggingface and @ModelScope2022.
Recipe: https://t.co/OhlAm9XYKB FP8: https://t.co/pzNsf3LBb9 BF16: https://t.co/igWS26BQ62
