Speech enhancement network improves recognition and sound quality
DualSpecSE: A Dual-Path Speech Enhancement Network Integrating Mel and Complex Spectrograms
Sound
Summary
Noise and poor audio quality can make understanding speech on devices hard. The authors created DualSpecSE, a system that cleans up speech by looking at sound in two ways: a simpler Mel representation for voice recognition and a detailed complex spectrogram for better sound quality. The system combines these two to produce clearer speech that works better for transcription and sounds more natural. Unlike some methods, it does this without needing extra tools to turn the cleaned-up version back into sound.
What this means in practice
- •For voice assistant developers: Improve transcription accuracy and audio quality by integrating dual-path speech enhancement models in noisy conditions.$Commercial implications: Enables voice assistants to deliver clearer responses and understand commands more reliably in diverse environments.
- •For telecommunications teams: Enhance call quality by applying dual spectrogram modeling to reduce noise and preserve natural speech during calls.
Authors
Xingchen Li, Ziqian Wang, Zikai Liu, Yike Zhu, Zihan Zhang, Longshuai Xiao, Lei Xie
Abstract
In this paper, we propose DualSpecSE, a speech enhancement framework that jointly models Mel-spectrogram and complex spectrogram in a dual-path architecture for improved ASR performance and higher-quality speech reconstruction. The Mel branch learns coarse-grained acoustic representations and produces enhanced Mel-spectrograms for direct ASR usage, while the complex branch refines fine-grained spectral details for high-fidelity waveform reconstruction. Built upon the cross-band and narrow-band blocks from CleanMel, DualSpecSE introduces an interaction module and a fusion module to enable effective information exchange between the two branches. The model simultaneously outputs enhanced Mel and complex spectrogram without requiring a pretrained vocoder. Experimental results demonstrate consistent improvements in speech fidelity, perceptual quality, and ASR performance. Codes and audio samples are available.