Swin transformers reduce model size and improve speech bandwidth extension
SwinDS-BWE: A Parameter-Efficient Swin-1D Lattice Generator with Decision-Science Discriminators for Speech Bandwidth Extension
Sound
Summary
Speech bandwidth extension aims to enhance audio by restoring missing high-frequency sounds. The authors designed a new model called SwinDS-BWE that uses a special kind of transformer to efficiently process audio and generate better high-frequency details. They also introduced three new small discriminators, inspired by decision science, to improve the model’s ability to judge and improve audio quality. This approach made the model smaller and more accurate than previous ones on English and French speech datasets.
What this means in practice
- •For audio software developers: Integrate a smaller and more accurate speech bandwidth extension model to improve audio quality in voice communication tools.$Commercial implications: This model enables more efficient and higher fidelity speech enhancement products for telecommunication and streaming platforms.
- •For embedded systems engineers: Deploy lightweight speech enhancement algorithms on devices with limited computational resources, such as hearing aids or mobile phones.
Authors
Tarikul Islam Tamiti, Sajid Fardin Dipto, Luke Baja-Ricketts, David Vergano, Anomadarshi Barua
Abstract
We propose SwinDS-BWE, a decision-science-inspired bandwidth extension (BWE) model with two coupled contributions: (1) Swin Transformers with lattice-style cross-stream interaction yield locality-aware modeling with linear-in-sequence per-window attention cost and reduce the prior AP-BWE generator size 0.5x from 33M to 17M. (2) Three novel lightweight decision-science-inspired discriminators augment AP-BWE's performance: a CVaR discriminator tail-pools activations to emphasize worst high-frequency (HF) segments, a Chance-Constraint HF discriminator penalizes excessive HF energy via a differentiable barrier, and a Multi-Criteria Utility discriminator learns convex style weights over spectral criteria. SwinDS-BWE surpasses prior AP-BWE with a 30x smaller discriminator (42.3M vs. 1.36M) and higher fidelity on English and French datasets. This work shows that decision-science-inspired critics can supervise BWE to reduce discriminator size.