FOAdapter enhances low-bitrate spatial speech coding for immersive audio
FOAdapter: A Plug-and-Play Spatial Tokenizer for Low-Bitrate FOA Speech Coding
Sound
Summary
Speech audio doesn’t just carry what people say but also where they are. This spatial information helps listeners tell speakers apart and feel like they’re in a real place. Existing speech coding methods work well for single-channel speech but lose these spatial details at low data rates. The authors propose FOAdapter, a tool that fits into current speech codecs and keeps these spatial cues intact even when compressing audio to very small sizes.
What this means in practice
- •For audio engineers: Encode spatial speech for conferencing systems preserving speaker location cues at low data rates.
- •For immersive media developers: Deliver natural spatial audio experiences with reduced bandwidth in virtual reality applications.
Authors
Kirak Kim, Yoonjeong Park, Juhan Nam, Sungyoung Kim, Minje Kim
Abstract
Spatially encoded speech conveys not only linguistic information but also spatial cues. This enables listeners to distinguish and localize speakers, contributing to more natural and realistic auditory experiences in applications such as remote conferencing and immersive media. Neural speech codecs (NSCs) enable high-quality reconstruction of monaural speech at increasingly low bitrates while preserving linguistic information, but their ability to effectively preserve spatial cues remains limited. We propose FOAdapter, a plug-and-play spatial tokenizer that leverages a pretrained monaural speech codec to efficiently represent both linguistic and spatial information in first-order ambisonic (FOA) speech at low bitrates. Comprehensive evaluation demonstrates that FOAdapter consistently preserves spatial information across all evaluated codecs. Source code and audio examples are available on our demo website.