IBM Granite 5.0 TurboCTC offers fast and accurate speech recognition
Design of the IBM Granite 5.0 TurboCTC ASR Model
Computation and Language
Summary
Transcribing spoken English quickly and accurately is a challenge. The authors designed a large model called Granite 5.0 TurboCTC that balances speed and accuracy well. It uses smart techniques to reduce processing time while keeping good recognition quality. They also trained it using public data and a new training method, making it easy to access and run efficiently.
What this means in practice
- •For speech technology developers: Integrate a fast and accurate English short-form speech recognizer optimized for real-time use in applications and devices.
- •For multilingual ai designers: Adapt the efficient model architecture and training strategies to develop speech recognition systems for other languages or dialects.
Authors
Brian Kingsbury, George Saon, Masayuki Suzuki, Hong-Kwang J. Kuo, Takashi Fukuda, Samuel Thomas, Vishal Sunder, Avihu Dekel
Abstract
We describe the architecture, training methodology and inference speedups of Granite 5.0 Turbo CTC, a 470 million parameter encoder-only model with an excellent speed-accuracy tradeoff. The architecture uses pyramidal temporal subsampling within Conformer blocks using strided depthwise convolutions, block-diagonal (chunk-wise) self-attention, and conditioning on intermediate predictions from the middle layer. Training highlights are the use of only publicly available data, the novel use of a Muon optimizer, and balanced data sampling. Inference speedups include replacing 1 x 1 convolutions with linear layers and optimizing the attention computation in the Conformer blocks. Collectively, these result in a model that is on the speed-accuracy Pareto frontier of the Open ASR leaderboard for English short-form ASR while being twice as fast as the fastest competitor. The model can be used under a permissive license and downloaded from https://huggingface.co/ibm-granite/granite-speech-5.0-470m-turboctc.