AI summaryⓘ
The authors study deep neural networks (DNNs) used for processing audio signals, focusing on their Lipschitz continuity—a property important for reliable behavior guarantees. They identify a problem: common audio DNNs that handle magnitude and phase separately are not covered by existing Lipschitz theories. To fix this, they introduce amplitude modifiers (AMs), a type of DNN that works only on magnitudes and provide conditions to ensure these AMs are Lipschitz continuous. They also design practical architectures and methods to measure their Lipschitz constants. Finally, they demonstrate how to use their approach in audio signal recovery, showing both theoretical convergence and real-world improvements in speech dereverberation.
Lipschitz continuitydeep neural networksaudio signal processingcomplex-valued signalsmagnitude and phaseamplitude modifierstime-frequency maskingplug-and-play algorithmsspeech dereverberationresidual maps
Authors
Kazuki Matsumoto, Ren Uchida, Natsuki Yoshino, Kohei Yatabe
Abstract
The Lipschitz continuity of deep neural networks (DNNs) is essential for establishing theoretical guarantees regarding their behavior. From both theoretical and practical perspectives, various methods have been proposed to construct Lipschitz-continuous architectures and control their Lipschitz constants. However, several DNN architectures common in audio signal processing fall outside the scope of existing theoretical frameworks, hindering the development of Lipschitz-continuous models in acoustic applications. In particular, despite their widespread adoption, DNNs that separately process the magnitude and phase of complex-valued signals cannot be Lipschitz continuous under existing frameworks. In this paper, to address this limitation, we establish a theoretical foundation for constructing amplitude modifiers (AMs), a class of DNN architectures that operate solely on the magnitude of a complex-valued input, with provable Lipschitz continuity. Specifically, we derive a necessary and sufficient condition for an AM to be Lipschitz continuous and propose LipsAMs (Lipschitz-continuous AMs) corresponding to common architectures for audio signals, including time-frequency masking. Furthermore, we develop an efficient framework for evaluating their Lipschitz constants and analytically derive these constants for some of the proposed architectures. As an application, we propose CoReM-LipsAM (Controlled Residual Maps via LipsAM) for plug-and-play (PnP) audio signal recovery, integrating a DNN as a data-driven prior within a model-based signal processing algorithm. The convergence of the obtained PnP algorithm is structurally guaranteed by the CoReM-LipsAM architecture and empirically validated through speech dereverberation experiments.