Anatomy-based adaptation improves vocal tract prediction across speakers
Anatomy-aware cross-speaker adaptation of complete vocal-tract acoustic-to-articulatory inversion
Artificial Intelligence
Summary
Predicting how people move their vocal tract when speaking usually struggles because everyone’s mouth and throat shapes differ. The authors created a way to adjust vocal tract predictions from one person to another by using key anatomical landmarks like teeth and spine positions. Their method changes the predicted shapes to fit a new speaker without needing to retrain the model. Tests show this approach lowers errors in representing vocal tract structures across different people.
What this means in practice
- •For speech technology developers: Adapt vocal tract shape prediction models to new speakers geometrically without retraining, improving personalized speech analysis or synthesis.
- •For speech therapy clinics: Use anatomy-aware models to estimate articulatory movements for patients without extensive speaker-specific data collection.
Authors
Nhat-Nam Nguyen, Pierre-Andre Vuissoz, Yves Laprie
Abstract
Cross-speaker acoustic-to-articulatory inversion requires accounting for anatomical differences between speakers. We propose a geometric adaptation framework that uses anatomical landmarks, primarily on vertebrae and dental structures,to transfer predictions from a fixed inversion model to unseen speakers. An affine transformation followed by thin-plate spline (TPS) deformation maps the predicted contours of 10 vocal-tract structures into each target speaker's geometry without retraining. Landmarks are identified in one selected /u/ frame per speaker as a common phonetic reference without assuming identical articulatory configurations across speakers, and the resulting mapping is reused across recordings. We train the model on a single-speaker rt-MRI database and evaluate adaptation on eight speakers from a separate multi-speaker rt-MRI database. We compare affine and TPS configurations using 12 or 14 landmarks. Affine12+TPS14 achieves the lowest mean point-to-closest-point error of 3.19mm. These results support the combined value of anatomical landmark information and nonrigid alignment.