Vietnamese speech and deepfake dataset with diverse dialects and code-switching
VietPrism: A large-scale Vietnamese speech and deepfake corpus with diverse dialects and code-switching
Computation and LanguageSound
Summary
Vietnamese speech research often faces challenges because data usually focus only on one problem at a time, like recognizing words or detecting fake audio. The paper introduces VietPrism, a large dataset that combines many speech features including different dialects, speaker identities, and Vietnamese-English mixing in conversation. It also includes more than 3,000 hours of artificially generated speech to test how well deepfake detectors work. The authors found existing detectors struggle when faced with voices that sound similar or with different dialects, showing that more robust tools are needed.
What this means in practice
- •For speech recognition developers: Develop and test Vietnamese speech recognition systems that handle multiple dialects and mixed Vietnamese-English language use.
- •For audio security teams: Create and evaluate tools to detect forged or synthetic Vietnamese speech across various dialects and speakers for security purposes.
Authors
Minh Hoang, Thai Le
Abstract
Vietnamese speech research is constrained by resources that isolate automatic speech recognition from speaker, dialect, code-switching, and deepfake analysis. We introduce VietPrism, an open, multi-domain corpus that brings these dimensions together at scale: 993.4 hours and 403,941 bona fide utterances from 1,262 verified speakers across 8,388 real-world videos. To our knowledge, it is the first large-scale Vietnamese corpus to jointly provide transcripts, consistent speaker identities, five dialect groups, and naturally occurring Vietnamese--English code-switching, which constitutes nearly half of the corpus by duration. We further create over 3.1K hours of spoof speech with four open-source and commercial synthesis systems. Every spoof is conditioned on a verified speaker reference and paired with a transcript- and speaker-matched bona fide utterance, enabling unique controlled evaluation with reduced lexical and identity confounds. Zero-shot evaluation of five pretrained multilingual detectors reveals striking brittleness: EER greatly varies across detector--generator pairings, while recent multilingual detector DFA-1B degrades from 16.3% to 33.6% as speaker similarity increases. Dialect-stratified results expose further model-dependent disparities. By unifying natural linguistic diversity with controlled spoof generation, VietPrism provides a challenging foundation for Vietnamese speech modeling and trustworthy audio-deepfake detection.