Low queer voice representation found in speech datasets and tensions explained
Queer inclusion in speech datasets: An audit and taxonomy of practical tensions
Artificial Intelligence
Summary
Many speech technology datasets include very few voices from queer people, less than 1.4%, which is not enough to study fairness properly. The authors looked closely at several speech datasets and also at datasets made with the queer community to understand why this is so. They found that common ways of collecting speech data often clash with values important to marginalized groups. The paper presents a list of these challenges to help future data collection better include queer voices.
What this means in practice
- •For speech technology developers: Assess and improve fairness by recognizing limitations in queer voice data representation in speech datasets.
- •For data collection teams: Design inclusive speech dataset collection methods that respect values important to queer and marginalized communities.
Authors
Brooklyn Sheppard, Anaelia Ovalle, Adina Williams, Levent Sagun
Abstract
In this paper, we examine speech datasets for their inclusion of LGBTQIA+, or queer, voices and provide a taxonomy of tensions to better understand why there is a lack of such voices in current speech technology datasets. Through an audit of six diverse speech datasets, we find that measurable queer representation is low (0-1.4% of speakers) - insufficient for robust disparity measurement. We take this community as a case study to consider what challenges and tensions are associated with collecting speech data from marginalized communities. For comparison, we audit an additional two datasets from the speech sciences that were created by, for, and with the queer community. We note that many customs in speech dataset collection efforts in AI and speech technology research may conflict with values emphasized in participatory approaches with marginalized communities, and provide a taxonomy describing these tensions.