Speaker verification model improves accuracy with efficient expert mixture design
MECT: Mixture of Experts with CNN-Transformer Network for Speaker verification
SoundArtificial Intelligence
Summary
Speaker verification means checking who is speaking by analyzing their voice. This paper presents a model called MECT that mixes different specialized parts called experts inside a combined CNN and Transformer network to better recognize speakers. The authors tested different ways to organize these experts and found improvements with only a small increase in model size. They also made MECT work well even for short voice chunks, which helps with streaming recognition.
What this means in practice
- •For voice assistant developers: Improve speaker recognition reliability in devices by integrating MECT for better accuracy on varied voice datasets with efficient computation.$Commercial implications: Enables more accurate and efficient speaker verification in commercial voice assistants serving diverse users.
- •For security system engineers: Deploy MECT-based models to authenticate users through voice, supporting real-time or streaming verification scenarios with low latency.
Authors
Yu Zheng, Jinghan Peng, ChangHao Zhang, Jian Liu, Weiqiang Wang
Abstract
In this paper, we propose MECT, a speaker verification model that integrates the Mixture-of-Experts (MoE) mechanism into a CNN-Transformer backbone with optimized block structure and stacking scheme. Specifically, we investigated four MoE variants that span utterance-level and frame-level granularity with dense and sparse routing strategies. The MoE mechanism proves to be effective over the baseline without MoE with only a small increase in parameters. We further scale MECT to a series of model sizes, all maintaining compact parameters and low computational complexity. In particular, MECT-B2 achieves state-of-the-art performance on VoxCeleb1 and delivers strong results on CN-Celeb, demonstrating its effectiveness across diverse datasets. In addition, we establish a streaming inference paradigm through causal retraining, which maintains strong performance at a chunk size of 100ms.