ARC 方法用稀疏重连替代稠密训练阶段，大幅降低稀疏化计算成本
AC/DC on a Budget - Alternating Sparse Phases
训练稀疏网络的省算力新招：用稀疏重连阶段替代昂贵的稠密阶段，CIFAR100、ImageNet 和 GPT-2 上都验证了效果，做模型压缩的可以看看。
传统 AC/DC 等稀疏化方法依赖交替进行稠密与稀疏训练阶段，其中稠密阶段计算开销大。论文发现稠密阶段的作用在于：重新引入的被掩码参数会获得异常大的梯度，驱动掩码重连和参数符号翻转。基于此提出 ARC（Alternating Rewiring Compression），包括三个改进：稠密阶段对被掩码参数零初始化、缩短稠密阶段、用只更新掩码和少量高梯度权重的稀疏重连阶段完全替代稠密阶段。在 CIFAR100 和 ImageNet 上，ARC 以远低于基线的计算量匹配 AC/DC 性能；GPT-2 在 OpenWebText 上的实验也显示验证困惑度和平均零样本性能提升。
AC/DC on a Budget - Alternating Sparse Phases
State-of-the-art neural network sparsification relies on alternating dense and sparse training phases. While computationally expensive, dense phases are critical for performance. Our empirical analysis reveals the mechanics behind this gain: upon reintroduction, previously masked parameters receive disproportionately large gradients that drive essential mask rewiring and parameter sign-flipping. Building on these insights, we optimize the performance-to-FLOPs Pareto frontier through three innovations: (1) zero-initializing masked parameters during dense phases to promote relevant sign flips, (2) shortening dense phases, and (3) replacing the dense phase entirely with a sparse rewiring phase that updates only the active mask and a small, high-gradient fraction of out-of-mask weights. Evaluated on CIFAR100 and ImageNet, our variant ARC (Alternating Rewiring Compression) matches baseline AC/DC performance at a fraction of the computational cost, establishing a superior performance-to-FLOPs frontier also compared to standard sparse-to-sparse algorithms. Experiments with GPT-2 on OpenWebText further show improvements in validation perplexity and average zero-shot performance
