Intermediate networks can be removed from multilingual speech parsing models
When Can You Prune Your Network? A Study of Intermediate Neurons in Multilingual Speech Parsing
Computation and Language
Summary
Speech parsing means turning spoken words into written words and understanding their grammar. The paper studies parts inside AI models, called intermediate neural networks, that help with this task. The authors show that by removing these intermediate parts, they can make the model smaller and still work just as well or better. They found these parts help only when some parts of the model are fixed and cannot learn. Their tests include languages like French, Slovenian, and Naija.
What this means in practice
- •For speech recognition engineers: Build smaller, efficient speech parsing systems that maintain accuracy by pruning intermediate networks when freezing pretrained layers.
- •For software developers for multilingual applications: Develop speech-enabled apps supporting multiple languages with reduced model sizes by removing intermediate components without losing performance.
Authors
Minnie Kabra, Benjamin Lecouteux, Maximin Coavoux
Abstract
End-to-end speech parsing, a task recently proposed, consists in predicting both the transcription and the syntactic tree for a spoken utterance. Existing architectures for speech parsing often utilise intermediate neural networks. In this work, we examine the effectiveness of intermediate neural networks (NN) for parsing, and, specifically, what role do they play. We introduce a simpler end-to-end architecture for speech parsing, where we remove these intermediate NN units, reducing the parameters by 12%, while achieving comparable or better performance than prior method on both automatic speech recognition (ASR) and parsing. We demonstrate that intermediate NN units help reduce the representational gap when the pre-trained encoder is frozen. We do a comprehensive evaluation of speech parsing on French, and medium-low resource languages Slovenian and Naija. We further investigate the impact of the training data size and intermediate layers of the pretrained speech encoder on speech parsing.