PhaseGAN improves vocoders by separating amplitude and phase reconstruction
PhaseGAN: High-Fidelity Vocoder via Decoupled Amplitude and GAN-Driven Phase Reconstruction
Sound
Summary
Creating realistic computer-generated speech involves accurately rebuilding sounds from data, which requires recovering both loudness and timing details. The authors identified that reconstructing the timing part, called phase, is a major challenge that affects sound quality and speed. They designed PhaseGAN, a small and fast system that builds sounds first by focusing on volume patterns, then separately reconstructs timing patterns using a special method. This approach makes the system produce better-sounding audio with fewer resources and even works for musical sounds it wasn’t trained on.
What this means in practice
- •For mobile app developers: Integrate a lightweight vocoder that runs in real-time on edge devices with high audio quality and low computational cost.
- •For game audio designers: Generate high-quality synthetic musical audio without requiring specifically trained musical data.
Authors
Wenzheng Zhang, Xueliang Zhang, Shulin He, Fei Zhao, Xin Liu, Pengjie Shen, Zhenlong Guo, Zixuan Xue, Hongtao Bao, Zixuan Li
Abstract
A vocoder is a pivotal component of modern text-to-speech (TTS) systems. Despite the significant progress of neural network-based vocoders, accurate phase reconstruction remains the main challenge limiting both audio quality and modeling efficiency. We introduce PhaseGAN, a lightweight vocoder that addresses this limitation through a "mel $\rightarrow$ Amplitude $\rightarrow$ Phase" reconstruction pipeline. By reconstructing amplitude and phase spectra via distinct methodologies, the proposed PhaseGAN outperforms state-of-the-art baselines while utilizing fewer model parameters and reduced computational requirements. The compact version generates high-fidelity audio with approximately 500K parameters and 1 GMAC computational load, making it highly suitable for real-time applications on edge devices. In addition, our approach exhibits exceptional musical audio synthesis capabilities despite no training on musical data, illustrating unprecedented cross-domain generalization. See https://github.com/phasegan/phasegan-audio-demo for demos of our work.