Summary
High-dimensional vectors are common in deep learning, but storing and processing them takes a lot of space and time. The authors propose a new method called SETAREH that changes both vectors in the same way to keep their inner product exact before reducing the data size. This saves storage and transmission costs by keeping only important parts while controlling how much approximation error is introduced. They tested this method on fake data and an age estimation task, showing it can work well with fewer data points. The benefits depend on how accurate the approximation needs to be and the cost of storing extra information about the transformations.
What this means in practice
- •For edge computing teams: Compress repeated high-dimensional input vectors to reduce memory and communication overhead in distributed edge AI systems using fixed weights.
- •For machine learning engineers: Reduce data size when repeatedly computing inner products with fixed model coefficients to speed up inference in neural networks.
Tested on one dataset.
Abstract
Modern deep learning systems increasingly rely on high-dimensional vector operations, in which large input and parameter vectors are repeatedly stored, transferred, and processed. Although substantial research has focused on reducing neural-network model size through pruning, quantization, knowledge distillation, and efficient architectures, the storage, memory-traffic, and communication costs associated with repeatedly presented high-dimensional input vectors remain important practical considerations, particularly in distributed and edge-computing environments. This paper proposes the SETAREH (Sorting-assisted Energy-concentrating Transform for Approximate Representation using Haar wavelets) transform, a Haar-wavelet-based sparsification approach for approximating the inner product between a fixed high-dimensional coefficient vector and successive input vectors. By applying the same sequence of permutations and Haar transforms to both vectors, SETAREH preserves the exact inner product before truncation. It then retains only a subset of the transformed coefficient-input pairs, reducing their representation cost while introducing a controllable approximation error. To evaluate SETAREH, we consider two complementary experimental settings: a synthetic Gaussian-vector baseline and a real apparent-age estimation task using a convolutional neural network on the APPA-REAL dataset. The results show that SETAREH can approximate the evaluated inner products using substantially fewer transformed coefficient-input pairs than the original vector dimension. The findings support its use as a potential structured data-representation method for repeated inner products with fixed weights, while the practical benefit depends on approximation accuracy and the cost of storing or transmitting permutation metadata.