Looped transformer improves audio recognition with fewer resources
LAST: Looped Audio Spectrogram Transformer
SoundMachine Learning
Summary
Deeper transformer models can understand audio better but need a lot of computing power and memory. The authors created a method called Looped Audio Spectrogram Transformer (LAST) that repeatedly refines only a small part of the model's output, making later steps cheaper to run. This approach improves audio recognition accuracy while using fewer parameters and computations. It also makes models more robust to sound disruptions and works well across different sound types like music and environmental noises.
What this means in practice
- •For audio software developers: Use LAST to build audio recognition systems that require less computing power without sacrificing accuracy, enabling more efficient real-time processing.
- •For sound monitoring teams: Deploy LAST-based models to improve recognition robustness in environments with noisy or masked audio inputs, enhancing event detection reliability.
Authors
Haider Al-Tahan, Sean O'Brien, Anastasia Razdaibiedina, N. Apurva Ratan Murty
Abstract
Increasing depth of transformer models improves recognition, but it comes at a substantial cost. Each additional layer requires more parameters, which makes the process computationally inefficient. We ask whether additional processing can focus on integrating features already computed. Looped Audio Spectrogram Transformer (LAST) first processes all tokens, then reuses the same blocks to refine only the class token over fixed audio features, thereby making later passes inexpensive. On AudioSet, ten-pass LAST achieves 0.345 mean average precision, exceeding a twelve-layer sequential transformer by 2.1% relative with 49.4% fewer parameters, 42% fewer multiply-accumulate operations, and 9.8% higher measured throughput. Across separately trained models, increasing the pass count from two to ten improves accuracy while adding only 1.2% computation. Further evaluations show improved robustness to temporal masking and various other auditory augmentations, with better generalization on classification tasks with music, environmental, and event sounds.