CRePE: Convolution-aware Relative Importance in Post-training Pruning with Efficient Search
2026-06-01 • Machine Learning
Machine Learning
AI summaryⓘ
The authors focus on making Large Language Models (LLMs) smaller and faster by removing unnecessary parts without extra training, a process called post-training pruning. They improve upon a previous method called RIA by adding a way to consider local 2D connections and adjusting importance weights more flexibly, naming their method CRePE. While finding the best settings for CRePE took a long time initially, they introduced a new faster search method called PHO that works well across different models. They also show that CRePE can be combined with other pruning techniques for even better results.
Large Language ModelsPost-training pruningRelative Importance scoringSparsityHyperparameter optimizationPerplexityModel compressionRow/column pruningChannel permutationNon-uniform sparsity
Authors
Cheonjun Park
Abstract
Deploying Large Language Models (LLMs) in practice incurs substantial memory and computational costs. Post-training pruning (PTP) is an effective approach to reducing these costs by removing weights without additional training. Among existing methods, RIA introduces relative importance scores normalized by row and column sums, achieving state-of-the-art accuracy. However, RIA considers only 1D cross-shaped (row/column) directional information and assigns equal weight to row and column contributions. In this paper, we propose \textbf{CRePE}, which incorporates 2D local neighborhood context and adaptive coefficients into Relative Importance scoring. CRePE consistently outperforms existing PTP methods across diverse models and sparsity settings. However, identifying optimal adaptive coefficients via perplexity (PPL)-based hill climbing requires numerous PPL evaluations and approximately 11 hours of search time. To address this, we propose \textbf{PHO} (Proxy-based Hyperparameter Optimization), which eliminates the need for repeated PPL measurements and reduces the search time to approximately 20 minutes. Furthermore, the optimal hyperparameter configuration found by PHO on one model transfers well to other models, demonstrating strong generalization. Finally, we verify that CRePE can be orthogonally combined with existing techniques including Channel Permutation, non-uniform sparsity allocation, and re-pruning methods.