MelT: GEMM-Native NDFT for Efficient Single-Stage Audio Frontends on Modern Accelerators
2026-05-31 • Sound
Sound
AI summaryⓘ
The authors developed MelT, a new way to process audio that replaces the usual two-step method with a single-step approach. Instead of using a Sparse Mel filter after a Short-Time Fourier Transform, MelT uses a special precomputed matrix to transform sound signals efficiently using dense matrix multiplication. This makes MelT much faster and uses less energy on different hardware while keeping the same accuracy in recognizing sounds. Their work focuses on making audio processing better suited for modern computers that work best with dense math operations.
Short-Time Fourier Transform (STFT)Mel scaleDiscrete Fourier Transform (DFT)Non-Uniform Discrete Fourier Transform (NDFT)General Matrix Multiplication (GEMM)audio frontendinference latencyenergy consumptionhardware accelerationaudio classification
Authors
Augusto Camargo, Marcelo Finger
Abstract
Modern audio processing networks are commonly deployed on accelerators whose peak throughput is obtained through dense linear algebra, whereas conventional acoustic frontends -- a Short-Time Fourier Transform (STFT) followed by sparse Mel aggregation -- remain structurally heterogeneous. This mismatch can introduce memory-bandwidth, dispatch, and intermediate-allocation overheads on contemporary accelerator backends. This work introduces MelT, a single-stage frontend framework in which Mel-spaced Non-Uniform Discrete Fourier Transform (NDFT) bases are precomputed and applied to time-domain acoustic frames through dense General Matrix Multiplication (GEMM) operations. The contribution is not the NDFT operator itself; rather, it is the formulation of Mel-spaced NDFT projection as a GEMM-native audio frontend and its evaluation as a hardware-efficient alternative to conventional STFT+Mel pipelines. Evaluated across platforms ranging from Apple A18 Pro edge hardware to NVIDIA H100 datacenter acceleration, MelT attains up to a $3.75\times$ speedup in inference latency and a $3.52\times$ reduction in energy consumption while maintaining downstream classification accuracy.