Acoustic progress boosts long-horizon decoding speed in speech recognition
Acoustic Progress Propagation for Long-Horizon Speculative Decoding in ASR
Computation and Language
Summary
Automatic speech recognition systems often decode speech one piece at a time, which can be slow. The authors developed a method that tracks how far decoding has progressed using sound information, helping the system guess future words more efficiently and speeding up the process. Their approach improves speed without losing accuracy, working better than previous methods especially when predicting longer sequences. They tested it on multiple speech datasets and made their code available for others to use.
What this means in practice
- •For voice assistant developers: Improve real-time transcription speed by integrating acoustic progress tracking into ASR decoding pipelines for faster user interaction.
- •For call center technology teams: Deploy faster automatic speech recognition for long conversations by reducing decoding time without compromising accuracy.
Authors
Yuanyuan Jia, Qianqian Yang
Abstract
Speculative decoding accelerates autoregressive automatic speech recognition (ASR), but the acceptance length of alignment-aware drafters can saturate as the draft horizon increases. We propose a progress-aware speculative drafter that recurrently propagates an acoustic progress state across draft steps and feeds it back into audio cross-attention to guide token generation. We jointly train the drafter and progress predictor over variable draft horizons. On five ASR test sets, our method achieves lossless, macro-averaged end-to-end speedups of 1.657x and 1.227x over target-only autoregressive decoding with Qwen3-ASR-0.6B and Qwen3-ASR-1.7B, respectively. Relative to AnchorDraft, our method improves the macro-averaged speedup by 34.3% and 9.0%, respectively. Horizon sweeps show continued growth in acceptance length beyond the baselines' saturation. Code is available at https://github.com/yuanyuanjia71-spec/ProgDraft.