Hybrid Compression: Integrating Pruning and Quantization for Optimized Neural Networks
2026-06-22 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors present a new way to make deep neural networks smaller and faster, which is important for using them on devices with limited memory and computing power. They first shrink the models using techniques like pruning (removing unnecessary parts) and quantization (simplifying data). Then, they combine several small compressed models using a method called Mixture of Experts to keep good performance while being efficient. Tests show their approach reduces the model size and computation needed, with very little loss in accuracy.
deep neural networksmodel compressionpruningquantizationMixture of ExpertsCNNFLOPsedge devicesmodel sizeinference efficiency
Authors
Minh-Loi Nguyen, Long-Bao Nguyen, Van-Hieu Huynh, Minh-Triet Tran, Trung-Nghia Le
Abstract
Deep neural networks have witnessed remarkable advancements in recent years and have become integral to various applications. However, alongside these developments, training and deployment of neural network models on embedding and edge devices face significant challenges due to limited memory and computational resources. These problems can be addressed with deep neural network compression, which involves a trade-off between model size and performance. In this paper, we propose a novel method for model compression through two phases. First, we utilize model compression techniques, such as pruning and quantization, to significantly reduce the model size. Then, we use Mixture of Experts to route the previously compressed models to enhance performance while maintaining a balance in inference efficiency. MoEs consist of multiple expert models (i.e., compressed models) that are moderately sized and deliver stable performance. Experimental results on several benchmark datasets show that our method successfully compresses CNN models which achieves substantial reductions in FLOPs and parameters with a negligible accuracy drop.