A Mathematical Theory of Reusable Neural Bases for Network Compression
2026-09-01 • Machine Learning
Machine LearningArtificial Intelligence
AI summaryⓘ
The authors address the problem of high memory use in large AI models by introducing a new design called the Linear Reusable Neural Bases Architecture (LRNBA). Their idea is to build each part of the network as a mix of shared smaller pieces, which saves a lot of memory and still trains well. This approach lets them create larger and deeper networks without using more parameters. Experiments show that their method trains as fast or faster and achieves lower error than traditional designs while staying stable.
neural networksparameter efficiencymemory costnetwork compressiontraining convergencerecurrent neural networks (RNN)model architectureinferenceloss function
Authors
Binshuai Wang
Abstract
As large AI models become increasingly prevalent across a wide range of applications, memory cost has become a critical bottleneck in both training and inference. To mitigate this issue, we introduce the Linear Reusable Neural Bases Architecture (LRNBA), a novel framework aimed at improving parameter efficiency and reducing memory cost. Inspired by recurrent neural network (RNN) designs, the core idea of our approach is to represent each network block as a linear combination of a shared set of neural bases, thereby enjoying highly network compression rate while maintaining stable training. The proposed architecture allows for the construction of significantly wider and deeper networks under the same parameter budget. Extensive experiments demonstrate that our model achieves comparable or even faster convergence and lower loss than classical architectures, while maintaining stable training dynamics.