受脑科学启发的剪枝框架 ASF-S:参数量减 70% 无需重训不掉精度
Artificial Structure Function Search: Preserving Artificial Functional Connectivity for Structured Pruning
arXiv 新论文:ASF-S 剪枝方法砍掉 70% 参数后不用重训就能恢复原精度,思路是从大脑的功能连接借来的。
arXiv 论文提出结构化剪枝框架 ASF-S,核心是名为 Principle Gradient Importance(PGI)的剪枝候选选择准则。PGI 借鉴大脑中结构与功能的关系,通过保留输出层的拓扑组织,为人工神经网络定义了人工功能连接(AFC)。在与近期基准的对比中,该方法得到参数量减少 70% 的模型变体。这些剪枝后的模型无需重训被剪层即可恢复基线精度。
Artificial Structure Function Search: Preserving Artificial Functional Connectivity for Structured Pruning
Structured pruning is a model compression technique that is used to reduce the computational cost of deploying deep neural networks on resource-constrained devices. Popular methods of pruning rely on opaque heuristics or weight-based criteria that give no indication as to the structural dependencies in the network. To address these limitations we present Artificial Structure Function Search (ASF-S): a novel structured pruning framework. ASF-S utilizes Principle Gradient Importance (PGI): a novel prune-candidate selection criteria that is inspired by structure-function relationships in the brain. By ensuring the pruned structure of the model respects topographical organization of the output layer, we define Artificial Functional Connectivity (AFC) for artificial neural networks. AFC provides evidence to demonstrate that accurate smaller networks can be found using careful prune candidate selection criteria. We present results for PGI as a selection criterion and for ASF-S as a pruning framework against recent benchmarks, demonstrating that our method yields model variants with 70\% parameter reduction, that can recover baseline accuracy without re-training the pruned layers.