Method reduces transformer computing and energy use by cutting inactive channels
Dense Structural Compression of Transformers via Gauge-Correct Channel Removal
Machine Learning
Summary
Transformers are powerful AI models, but they use a lot of energy because of heavy calculations. The authors propose a method that trims parts of these models that don’t help much during processing, making them faster and more efficient without losing accuracy. They introduce a new way to measure which parts to remove, fixing previous issues with stability. Their approach leads to smaller, faster models that still perform well on language and mathematical tasks.
transformerinferencedense matrix multiplicationchannel pruninggauge freedomgroup lassocompute efficiencypolynomial long divisionmasked autoencodingmodel compression
Authors
Jed A. Duersch, Naïm Es-Sebbani, Nathanaël Haas, Zied Bouraoui
Abstract
Inference energy per token drives the cost and carbon footprint of deployed transformers. It is dominated by dense matrix products that incur fused multiply-accumulate (FMA) operations and memory traffic. To reduce these computations while retaining dense tensors for high GPU throughput, we develop a methodology from first principles to adapt structural complexity during training to maximize inference utility per unit compute. Channel penalties drive entire tensor slices to zero to enable physical removal while preserving density and the network function. The natural approach, penalizing the norm of operator components acting through each channel, is provably destabilized by gauge freedom. We resolve this pathology with GaugeLasso: additive symmetric group-lasso penalties that recover a monotone function of product-norms when the network converges to gauge balance. Our equilibrium analysis enables per-channel calibration to correctly suppress slices that under-perform in inference utility per unit compute. Under adaptive pressure, the network reorganizes into depth-dependent structural profiles that can be far smaller than the architecture required to learn the task. On polynomial long division over $\mathbb{F}_{31}$, compute compresses from 148 to 255 times with perfect accuracy. On character-level language modeling, compressed models outperform the hand-designed baseline at equal FMA. On masked autoencoding, a compression trial exposes which axes were over-provisioned and which saturated, guiding a better second design. Compaction also accelerates training monotonically as the model progresses. Post-hoc pruning with the same utility ranking cannot reach these structures, showing that sustained pressure is central to discovery of efficient models. Retraining a discovered architecture recovers baseline quality on our statistical tasks, but fails on our exact algorithmic task.