Online algorithms improve real time speech extraction with moving speakers
Online Algorithms for Independent Low-Rank Matrix Analysis and Rank-Constrained Spatial Covariance Matrix Estimation Based on Maximum Weighted Likelihood Estimation
Sound
Summary
Extracting clear speech from multiple microphones in noisy places is hard, especially when people move. The authors developed new online methods that update their understanding of the sounds quickly and work better for moving speakers. Their approach uses clever math to update estimates faster for real-time use. Tests with both simulated and real recordings show their methods produce clearer speech than older techniques.
What this means in practice
- •For hearing aid developers: Improve speech clarity when users or speakers move by updating signal separation in real time under noisy conditions.$Commercial implications: This paper makes it possible to sell hearing aids that better extract speech dynamically as users or environments change.
- •For speech recognition engineers: Enhance real-time speech extraction algorithms to handle moving speakers for more accurate voice input in noisy environments.
Authors
Yuto Ishikawa, Norihiro Takamune, Tomohiko Nakamura, Daichi Kitamura, Hiroshi Saruwatari, Yu Takahashi, Kazunobu Kondo
Abstract
Real-time multichannel speech extraction (MSE) under diffuse noise conditions is an important task with a wide range of applications, such as speech recognition and hearing aids. In this paper, we propose online algorithms for independent low-rank matrix analysis (ILRMA) and rank-constrained spatial covariance matrix estimation (RCSCME). Previously, we proposed a real-time extension of the RCSCME-based method: an MSE method based on ILRMA and RCSCME using the blockwise batch algorithm. However, it assumes that the spatial characteristics are stationary within a single batch, and thus, in dynamic situations where the target speaker moves, its performance may degrade. To address this problem, we derive the online algorithms for ILRMA and RCSCME in the following three steps. First, we formulate framewise cost functions for ILRMA and RCSCME on the basis of maximum weighted likelihood estimation. Second, we derive the update rules for the framewise cost functions on the basis of auxiliary-function techniques. These naive update rules are computationally costly for real-time execution on a practical machine. Thus, we finally derive the online algorithms by approximating some intermediate parameters with their estimates. Furthermore, we propose stabilization and further acceleration techniques for these online algorithms. In experiments, we simulate situations where a target speaker is stationary or moves and show that the proposed method achieves superior speech extraction performance compared with conventional methods. In addition, using real-world recorded signals, we demonstrate the effectiveness of the proposed method in practical scenarios.