OpenPA assesses pronunciation accurately without forced alignment
OpenPA: Forced-alignment-free Multi-granular Pronunciation Assessment with Mixed-transcript Adaptation
Computation and LanguageSound
Summary
Pronunciation assessment tools often need detailed matching of spoken words to text, which can be complicated and limited in certain settings. The authors propose OpenPA, a new system that predicts how well someone pronounces words without requiring this strict matching step. OpenPA works well for both scripted and free responses, handling different types of transcripts in one model. It was tested mostly on one dataset but also performed well on others, including speech from people with oral cancer, showing its wider potential.
What this means in practice
- •For language learning app developers: Integrate better pronunciation scoring for both scripted and open-ended speech without relying on forced alignment, improving user feedback.$Commercial implications: This enables commercial language learning apps to provide more flexible and accurate pronunciation feedback, enhancing user engagement and effectiveness.
- •For speech therapy software teams: Use OpenPA to assess pronunciation in diverse speech types including pathological speech like oral cancer patients without changing models.
Authors
Fu-An Chao, Bi-Cheng Yan, Berlin Chen
Abstract
Recent advances in automatic pronunciation assessment (APA) have driven a shift from scripted to free-form assessment. However, existing methods often rely on multi-stage pipelines with forced alignment and separate feature extractors, limiting their practicality across response settings. In light of this, we propose OpenPA, a novel APA framework that leverages Whisper's cross-attention mechanism to jointly predict word- and utterance-level pronunciation scores without forced alignment. Through mixed-transcript adaptation and adaptive substitution weighting, OpenPA accommodates heterogeneous transcripts across closed- and open-response settings within a single model. Trained solely on speechocean762, OpenPA achieves the strongest overall performance among evaluated systems under in-domain closed and simulated open-response conditions, while maintaining its performance zero-shot on MultiPA. An additional cross-task evaluation on oral-cancer speech suggests its potential applicability beyond non-native speech.