Neural networks improve audio codec efficiency and quality with low delay
Lightweight Neural Prediction for MDCT-Based Perceptual Audio Coding
Sound
Summary
Audio codecs compress sounds by predicting parts of the audio so less data needs to be sent. The authors add a small neural network inside a traditional audio codec to better guess the next audio segment, which helps reduce leftover errors and improve sound quality. Their neural network, especially a type called a convolutional neural network, works better than older methods and small networks, achieving higher clarity without slowing down playback. This method keeps processing fast and uses fewer parameters, meaning it can work well on everyday computers.
What this means in practice
- •For audio codec developers: Integrate lightweight neural predictors to reduce coding delay while improving audio quality in standard audio compression systems.
- •For mobile device engineers: Implement efficient neural prediction to enhance audio playback quality on devices with limited computing power and low latency requirements.
Authors
Niccolo Abate, Gabriel Soule, Marina Bosi
Abstract
Perceptual audio codecs often use predictive coding to further reduce signal redundancy. Rather than learning the entire codec pipeline end-to-end, we insert a small neural network as a drop-in, deterministic predictor in an otherwise conventional codec: the network estimates the next Modified Discrete Cosine Transform (MDCT) coefficient block from a short context of reconstructed blocks, and the encoder transmits the quantized residual. We compare Multilayer Perceptron (MLP) and Convolutional Neural Network (CNN) predictors against a non-predictive baseline and a per-line Normalized Least Mean Squares (NLMS) linear predictor (similar to the MPEG-2 AAC Main Profile predictor). The CNN consistently outperforms the MLP, the linear predictor, and the baseline in signal-to-noise ratio (SNR), noise-to-mask ratio (NMR), and MUSHRA evaluation scores. Using a 512-sample analysis window (w = 512), our neural predictors recover, and the CNN exceeds, the coded quality obtained by the non-predictive baseline after quadrupling the analysis-window length to w = 2048, at substantially lower algorithmic delay. The CNN uses only 0.13M parameters (roughly 20x fewer than the MLP) and has a decoder real-time factor (RTF) of ~0.10 on a consumer CPU.