Audio recognition improves using high-res training with low-res testing
RAST: Resolution-Aware Privileged Structure Transfer for Low-Resolution Audio Activity Recognition
SoundArtificial Intelligence
Summary
Recognizing human activities from sound usually works better when audio is recorded at high quality, but high-quality audio uses more energy and may capture private speech. The authors focused on using only low-quality audio when recognizing activities but trained their system with both high- and low-quality sounds. Their method, called RAST, helps the system learn important sound details from high-quality recordings and apply that knowledge to low-quality recordings. This approach improved recognition accuracy by up to nearly 8% compared to training only on low-quality audio.
What this means in practice
- •For mobile app developers: Develop audio-based apps that recognize activities with low-energy, privacy-preserving microphones by training models on high-quality audio data.$Commercial implications: Enables privacy-aware activity recognition apps that operate with low-resource audio inputs, appealing to consumer markets focused on privacy and battery life.
- •For smart home device engineers: Build devices that infer occupants’ activities accurately using low-resolution audio sensors by leveraging training methods that incorporate high-resolution sounds.
Authors
Ji Hwan Park, Gautham Krishna Gudur, Yufei Shen, Dawei Liang, Edison Thomaz
Abstract
Audio is increasingly used for human activity recognition (HAR) because it captures object interactions, environmental events, and contextual cues in everyday environments. High-resolution (HR) audio provides rich acoustic information for model development but incurs substantial energy and storage costs and may expose sensitive speech content. Low-resolution (LR) audio offers a more privacy-preserving and resource-efficient alternative for deployment, but reduced sampling rates can remove acoustic cues essential for activity recognition, leading to significant performance degradation. We formulate this training-deployment mismatch as sensor-resolution privileged learning, in which HR audio is available during training, while inference relies exclusively on LR audio. We propose RAST, a resolution-aware transfer framework that compresses HR teacher representations by preserving token-level information and neighborhood structure before performing localized HR-LR alignment. Experiments on the SAMoSA and AudioIMU datasets show that RAST consistently outperforms LR-only training and direct teacher-transfer baselines, improving LR-only recognition by up to approximately 7.8% while requiring only LR audio at inference.