AMD 提出 SliMOO：用多目标进化搜索做 LLM 深度剪枝
SliMOO: Interpretable Multi-Objective Evolutionary Search for LLM Depth Pruning
AMD 把剪枝当成多目标优化来做，Llama-3.1-70B 砍掉 25% 层后数学分还能涨 10 个点，代码也会开源。
AMD 研究团队提出 SliMOO，把 LLM 深度剪枝建模为多目标优化问题，先用单层移除探测构建任务-层重要性图，再用 NSGA-II 进行 Pareto 感知的进化搜索。在 Llama-3.1 和 Qwen3 两个模型家族上，SliMOO 的多任务权衡表现优于规则式、贪心和标量化基线。在 Llama-3.1-70B、25% 稀疏度下，数学域平均分从 24.33% 提升到 34.40%，代码域平均分从 35.03% 提升到 39.14%。代码将开源于 GitHub 的 AMD-AGI/SliMOO 仓库。
SliMOO: Interpretable Multi-Objective Evolutionary Search for LLM Depth Pruning
As large language models (LLMs) grow more powerful, their scale makes them costly to serve. Depth pruning offers a practical route to efficiency by directly reducing memory and latency without hardware-specific support. However, we observe that transformer layers exhibit markedly different task-dependent layer contributions, whereas existing depth-pruning methods ignore this heterogeneity and thus tend to optimize for a narrow domain or benchmark rather than preserve broad capability. This suggests that LLM depth pruning should be formulated as a multi-objective optimization problem rather than a single-score ranking problem. We propose SliMOO, an interpretable multi-objective evolutionary search framework for LLM depth pruning. SliMOO first uses single-layer removal probing to construct a task-layer importance map, which provides an interpretable view of task-shared and task-specific layers and serves as a search prior. It then performs a Pareto-aware evolutionary search for pruning masks, where candidates are generated under the guidance of the task-layer prior, scored by their deviation from the dense model on each task, and retained through NSGA-II environmental selection. Across the Llama-3.1 and Qwen3 model families, SliMOO consistently achieves stronger multi-task trade-offs than rule-based, greedy, and scalarized baselines. For example, on Llama-3.1-70B at 25% sparsity, it improves the math-domain average from 24.33% to 34.40% and the code-domain average from 35.03% to 39.14%. Code will be made available at https://github.com/AMD-AGI/SliMOO.