Variable length coding improves neural network compression and accuracy
VLCQ: Post-training quantization for deep neural networks using variable length coding
Computer Vision and Pattern RecognitionArtificial IntelligencePerformance
Summary
Deploying deep learning models on small devices is hard because the models are large and require a lot of computing power. The authors present VLCQ, a way to shrink these models by using flexible, variable-length codes instead of fixed-size numbers. This method reduces the size of the model significantly while keeping accuracy high, without needing the original training data or extra retraining. They tested this on tasks like image recognition and object detection, showing better compression and performance than other post-training methods.
What this means in practice
- •For mobile app developers: Shrink deep learning models for mobile apps to run efficiently on limited hardware without losing much accuracy.
- •For embedded system engineers: Compress neural networks for use in embedded devices where memory and power are limited, enabling better on-device AI.
Authors
Reem Abdel-Salam, Ahmed H. Abdel-Gawad, Amr G. Wassal
Abstract
Quantization plays a crucial role in efficiently deploying deep learning models on resource-constrained devices. Post-training quantization does not require access to the original dataset or retraining the full model. Current methods that achieve high performance close to baseline results typically require INT8 fixed-point integers. However, achieving higher model compression with lower bit-widths often leads to significant performance degradation. In this paper, we propose VLCQ, a post-training quantization method that relaxes the fixed-point encoding constraint, enabling more flexible exploration of quantization techniques through variable-length encoding. This approach achieves competitive or improved performance while reducing bit-widths, without requiring access to training data or model fine-tuning. Extensive experiments on image classification, segmentation, and object detection demonstrate that VLCQ improves model compression and accuracy compared with state-of-the-art post-training quantization approaches. For per-channel quantization, VLCQ surpasses FP32 accuracy and Piece-Wise Linear Quantization (PWLQ) on most evaluated models, achieving up to 6x compression compared with FP32 and up to 1.7x compared with PWLQ. When prioritizing compression with limited accuracy loss, VLCQ achieves up to 12.25x compression compared with FP32 with less than 4% performance degradation. For per-tensor quantization, VLCQ is competitive with Data-Free Quantization (DFQ), while providing greater flexibility to achieve lower bit rates across different tasks and models.