MicroQonv：面向卷积层的微缩放量化训练与推理加速方法
MicroQonv: Reshaping Convolution Tensors for Efficient Microscaling in Training and Inference
一篇做低比特量化的论文，专治卷积层里 im2col 导致激活体积暴涨的问题，量化开销最多降 9 倍，做边缘端部署的可以看看。
arXiv 论文提出 MicroQonv，将微缩放（microscaling）量化与卷积层的前向和反向传播结合，每个张量只量化一次，并在 im2col 变换前先量化激活张量。相比直接对全精度张量量化的做法，权重和梯度的量化开销降低 2 倍，激活张量最多降低 9 倍。该方法将内存搬运和存储相比全精度方案最多减少 7.53 倍，在 YOLOv8nano 和 YOLOv26nano 目标检测模型上分别把量化激活的内存搬运减少 3.5 倍和 2.2 倍。在边缘端持续学习的量化潜变量回放策略中，还支持 4-bit 微缩放，准确率提升 5.7% 到 11%。
MicroQonv: Reshaping Convolution Tensors for Efficient Microscaling in Training and Inference
Microscaling quantization techniques are increasingly used to represent neural network parameters with 8 bits or fewer while preserving near-full precision accuracy. However, applying these methods efficiently in convolutional layers is not straightforward. A naive approach transfers full-precision weights and activations to processing units and quantizes each tensor twice, resulting in much more memory movement than expected. Additional overhead comes from the activation tensors, whose sizes grow substantially because of the im2col transformation applied before quantization. We propose MicroQonv, a way to combine microscaling with convolutional layers' forward and backward operations by quantizing each tensor only once and quantizing the activation tensor before applying a modified version of im2col: channel-batch-first im2col. MicroQonv reduces the quantization cost by a factor of $\times2$ for weights and gradients, and by up to $\times9$ for activations, at a negligible accuracy cost. It reduces memory movement and storage by up to $\times7.53$ compared to their full-precision counterparts. This way, MicroQonv reduces microscaling-quantized activation memory movement by $\times3.5$ for state-of-the-art object detection models YOLOV8nano and $\times2.2$ for YOLOV26nano. It also enables 4-bit microscaling in a quantized latent replay strategy for continual learning at the edge, improving accuracy by +5.7% to +11%.
