Speech dataset created to fairly test voice anonymization tools
VoxAnon: A balanced dataset for a fair evaluation of speech anonymization
Sound
Summary
Speech anonymization tools try to hide who is speaking, but tests can be unfair if the test voices aren’t diverse. The authors created VoxAnon, a balanced speech dataset with equal representation of gender, age, accent, and pitch. This helps show which groups are harder to anonymize and ensures privacy tests aren’t biased. They found that even with balanced data, some differences remain depending on voice features and the tools used.
What this means in practice
- •For voice privacy developers: Test voice anonymization tools on balanced data to fairly evaluate privacy across different speaker groups.
- •For call center technology teams: Improve anonymization techniques by identifying residual privacy risks related to accent and pitch in caller voices.
Authors
Orane Dufour, Paul Magron, Mickael Rouvier
Abstract
Speech anonymization is evaluated with automatic speaker verification attackers whose performance depends on the attribute distribution of both their training and evaluation data, which may bias privacy estimates for some speaker groups. To alleviate this issue, we introduce VoxAnon, a dataset built from VoxCeleb1 and 2 with training and evaluation splits balanced across gender, age, accent, and pitch. We extract and cross-check attributes using two complementary methods, balance trial lists per subgroup, and draw test/enrollment utterances from distinct video sources in order to limit linguistic and session bias. Comparing VoxAnon with unbalanced training and test data across four attackers and three anonymizers, we show that attribute imbalance changes which subgroups appear hardest to re-identify, in directions that depend on the attribute. In the balanced setting, residual disparities persist, mostly tied to accent and pitch, and depend on the anonymizer/attacker pair. Code and trial lists are publicly available.