Bridgecodec enables audio decoding between mismatched codecs at ultra low bitrate

BridgeCodec: Mamba Enhanced Neural Audio Codec with Schrödinger Bridge at Low Bitrate

SoundArtificial IntelligenceMachine Learning

Summary

Audio codecs that compress and decompress sounds usually only work with paired components designed together, which limits mixing different systems. The authors introduce BridgeCodec, a method that lets different audio codecs talk by converting their compressed forms even if they weren’t originally paired. They use a special mathematical technique called Schrödinger Bridge to accurately translate these compressed audio representations without needing prior assumptions about noise. Their system also uses an enhanced neural network design to better understand long stretches of speech. Tests show BridgeCodec can remake audio at very low bitrates with good sound quality, even when bridging very different codecs.

What this means in practice

Authors

Zijian Lin, Jing Yang, Jinghao Luo, Zhuo Wang, Fan Fan, Zhiyong Wu

Abstract

Neural audio codecs achieve remarkable performance but operate as closed ecosystems with rigidly co-trained encoderdecoder pairs, severely hindering interoperability. To resolve this, we propose BridgeCodec, a versatile framework that decouples these pairs by translating latent representations between mismatched, frozen endpoints. We formulate this cross-codec translation as a Schrödinger Bridge optimal transport problem, enabling robust mapping between disparate distributions without predefined noise priors. To efficiently capture longrange speech temporal dependencies, we integrate a Mambaenhanced U-Net architecture. We validate BridgeCodec on an extreme deployment scenario: mapping a lightweight 8 kHz source encoder directly to a 48 kHz target decoder at an ultralow bitrate of 1 kbps. Experimental results demonstrate that BridgeCodec achieves superior wideband reconstruction and high perceptual quality, successfully bridging heavily bottlenecked, heterogeneous codecs.
View PDFOpen arXiv