Speech recognition runs fast on tiny low power microcontrollers
Streaming Speech Recognition on a Microcontroller: a 65.7M-Parameter Transducer in 2 and 4 Bits on the ESP32-P4
Hardware ArchitectureMachine Learning
Summary
Running powerful speech recognition software usually needs big computers, but this work shows it can work well on a small, low-power microcontroller chip. The researchers compressed a large speech model to use very little memory without extra training, so it fits and works accurately on the chip. Their setup processes short chunks of audio faster than real time with good accuracy on several speech tests. This work makes it easier to add speech recognition to tiny devices without special hardware accelerators.
What this means in practice
- •For embedded system developers: Enable real-time speech recognition on low-cost microcontrollers for voice-activated embedded devices.
- •For mobile device engineers: Integrate offline speech transcription in resource-constrained handheld devices without needing specialized neural accelerators.
Authors
Billal Boumedine
Abstract
We run Kroko, a 65.7M-parameter streaming Zipformer2 transducer for English, on the ESP32-P4, a microcontroller with two 400 MHz RISC-V cores, a 128-bit SIMD extension (PIE) and 32 MB of PSRAM, but no neural accelerator. Without any training, the encoder's 290 matrices are stored with 2 or 4 bits per weight in Hadamard-rotated groups: the width of each matrix comes from its measured sensitivity, and each matrix is first corrected toward the original model's output, then rounded with GPTQ and a search over the group norms. The released 71 MB of int8 weights become a 26.65 MB file. Activations are int16 between the matrix products and int8 into them. A C engine with hand-written PIE kernels processes each 1.28 s chunk of audio in 0.90 s on both cores, beam search included, and its PC build computes the same bits as the chip, so accuracy is measured on whole test sets. LibriSpeech WER is 4.34% on test-clean and 10.26% on test-other, against 3.94% and 9.17% for the released model; on FLEURS and Common Voice the gap grows to 2.3 and 1.1 points. A GPU simulator shows that the integer arithmetic costs nothing and that the whole gap comes from the weights, and an ablation on all of test-clean shows that GPTQ does most of the work. We also report the port's speed history, from 51.3 s to 0.62 s of encoder time per chunk, each design step measured again in one build, an attempt to reach 16 MB that stopped at 19.63 MB, and what did not work. We did not measure energy. The inference engine, the firmware and the 2/4-bit weights are available at https://github.com/BoumedineBillal/esp32-pie-asr.