Geometry aware system improves multi microphone speaker extraction
Microphone Geometry Aware Regional Target Speaker Extraction
SoundArtificial Intelligence
Summary
Extracting a person's voice from audio recorded by multiple microphones usually needs exact speaker location, but this new approach only needs to know roughly where they are. The authors designed a system that adapts to different shapes and numbers of microphones, so it works even if the setup changes. Their method uses a special attention model to understand how sound arrives at each microphone and improves this understanding using spatial reconstruction. Tests show it performs better than previous similar methods and nearly as well as those tuned for a fixed microphone layout.
What this means in practice
- •For audio equipment designers: Develop microphone arrays and signal processing pipelines that adapt to different shapes and numbers of microphones for improved voice extraction in diverse setups.$Commercial implications: Enables flexible multi-microphone devices that isolate speakers better, valuable for consumer electronics and communication tools.
- •For conference room technology teams: Improve voice clarity in meeting rooms by deploying speaker extraction systems that perform well despite changing microphone layouts.
Authors
Shitong Xu, Niki Trigoni, Andrew Markham
Abstract
Regional target speaker extraction (RSE) aims to extract speech originating from a specified angular region, relaxing the requirement for precise target speaker location when performing location based speech extraction. However, existing RSE methods typically assume a fixed microphone array configuration and microphone number, limiting their applicability to unseen microphone geometries and varying numbers of microphones. In this work, we propose a geometry-aware RSE framework capable of extract with varying microphone array geometry and microphone numbers. Specifically, we introduce an attention-based multichannel front-end to encode inter-microphone phase information, together with a spatial audio reconstruction-based auxiliary objective to improve the front-end spatial representations quality. Experimental results demonstrate that the proposed framework improves SI-SDRi by 5.1 dB over geometry-aware baselines across microphone arrays with four-microphones, while maintaining a performance gap of less than 0.6 dB compared with a tetrahedral geometry-specific baseline when evaluated on the tetrahedral microphone array.