I-Parakeet runs accurate speech recognition fully on mobile integer hardware
I-Parakeet: Integer-Only Conformer ASR on Mobile NPU
Computation and LanguageSound
Summary
Speech recognition models that understand spoken words usually require powerful computers because they use many calculations involving decimals. The authors created I-Parakeet, a version of one such speech model that works only with whole numbers, allowing it to run very fast on smartphone chips designed for integer math. They solved tricky math challenges to make this possible without losing much accuracy. This means speech recognition can run faster and more efficiently on devices like phones without relying on slower or power-hungry parts.
What this means in practice
- •For mobile app developers: Run accurate speech-to-text models directly on smartphones using only integer math, improving speed and energy efficiency without fallback to CPUs.
- •For embedded system engineers: Integrate Conformer-based speech recognition on integer-only NPUs in devices with strict power or hardware constraints.
Authors
Taichi Nishimura
Abstract
In this paper, we propose I-Parakeet, an integer-only implementation of NVIDIA's Parakeet-CTC (0.6B parameters) that runs on a smartphone NPU without any floating-point operator or CPU fallback. Modern Conformer ASR models are hard to deploy on edge devices because of their size, and quantized models still fall back to floating point for numerically sensitive operations. This prevents them from fully exploiting integer accelerators such as mobile NPUs. To achieve this, our contributions are threefold. First, we derive an integer formulation of the relative-positional self-attention at the core of the Conformer. We fuse its two score branches with different quantization scales and the relative shift into integer-only operations. Second, we introduce a minimax-optimized Swish approximation that minimizes the maximum error of the Swish output. Third, a layer-wise range analysis of activations yields two targeted remedies: an INT16 grid for the BatchNorm output and percentile calibration for the heavy-tailed pre-encoder activations. I-Parakeet achieves 4.97% WER on LibriSpeech test-other, running on a Qualcomm NPU at a real-time factor of 0.048, 7.5x faster than a CPU baseline.