腾讯论文提出 RAMPr：剪 MoE 模型内弱神经元，保留 91%-98% 性能
搞 MoE 部署的工程师可以看看这篇。腾讯的方法不是删整个专家，而是按 RAMP 分数剪专家内部的弱神经元，剪掉一半后显存省 36%-43%，吞吐还涨 17%-39%，比整专家剪枝更能保住数学和代码能力。
腾讯发布论文，提出针对 Mixture of Experts 模型的 RAMPr 剪枝方法，核心结论是：砍掉最不有用的 25% 专家神经元后，模型在 6 个被测模型上仍保留 91% 到 98% 的原始性能。
这与传统做法不同——以往压缩通常把整个专家当作单元删除，而该论文证明数学、编程等领域技能实际分散在多个专家内的具体神经元中。论文用名为 RAMP 的指标衡量每个神经元的被路由贡献：同一领域的神经元在不同样本间重叠度达 58% 到 74%，跨领域仅 15% 到 24%，且这些神经元分布在混合专家中，说明专家并非按领域清晰划分。
剪掉 50% 神经元时，GPU 显存占用下降 36% 到 43%，批量 prefill 吞吐量提升 17% 到 39%。
New Tencent paper shows domain skills (like math or coding ) in Mixture of Experts models sit in scattered neurons across expert.
so cutting weak neurons instead of whole experts shrinks the models while keeping most skills.
In their experiment, Mixture of Experts AI models kept 91% to 98% of their performance after their least useful 25% of expert neurons were cut.
MoE models send each token to a few experts, which suggests each expert handles a domain like math or code. Other work finds routing mostly tracks token type instead.
They scored each neuron's routed contribution with a measure called RAMP. Neurons picked for one domain overlapped 58% to 74% across samples of that domain, versus 15% to 24% across domains, and sat in mixed experts.
RAMPr prunes by these scores across all experts and kept 91% to 98% of original performance with 75% of neurons left, across 6 models. With 50% left, GPU memory fell 36% to 43% and batched prefill throughput rose 17% to 39%.
If you compress MoE models, prune neurons inside experts by their routed contribution instead of treating each expert as a single unit.
本条情报归属 AITOP「论文」核心栏目，重点聚焦中国 AI 产业与国产大模型生态。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：Mixture of Experts、腾讯、模型压缩、RAMPr，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「rohanpaul_ai」（@rohanpaul_ai），经 AITOP 多维评分管线结构化提炼。
