Speech recognition runs on a $5 microcontroller without special hardware
Oído: Open-Vocabulary Speech Recognition on a $5 Microcontroller without a Neural Accelerator
SoundComputation and Language
Summary
Speech recognition typically needs powerful processors or fixed commands on inexpensive microcontrollers. The authors created Oído, a system that runs English speech recognition on a cheap microcontroller without any special neural hardware. It can understand an open set of words with accuracy similar to larger, more complex models. While it’s not quite real time yet, it produces accurate transcriptions and supports partial results as people speak. The system even works well under noisy conditions and is better than some similar models running on laptops.
What this means in practice
- •For embedded device developers: Run open-vocabulary English speech recognition on low-cost microcontrollers without dedicated neural accelerators.
- •For consumer electronics manufacturers: Build voice-controlled devices that can transcribe natural language commands accurately using a $5 microcontroller.$Commercial implications: Enables low-cost smart devices with advanced speech features, reducing hardware requirements and increasing accessibility.
Authors
Daniel Varela Sánchez, Guillermo Martín-Coello Juárez
Abstract
Speech recognition on low-cost microcontrollers is, in practice, limited to keyword spotting or a fixed list of commands. We present Oído, an inference engine that runs open-vocabulary English speech recognition entirely on an ESP32-S3 (240 MHz dual-core Xtensa LX7, 512 KB of SRAM, 8 MB of PSRAM, 16 MB of flash, no neural accelerator). Oído executes a 13M-parameter Conformer-CTC model with int8 weights, per-frame dynamic int8 activations and int8 attention on the chip's SIMD instructions, with a schedule designed around flash and PSRAM bandwidth. On LibriSpeech test-clean/test-other the on-device arithmetic reaches 3.70%/8.23% word error rate (WER), within 0.12 points of full precision, and 3.33%/7.15% with an on-chip 1.3M-parameter language model; a transducer head gives 2.99%/6.67% and an int4 variant fits in 8.3 MB. This is 2.3-2.8x fewer errors than the vendor's recognizer on the same chip, and fewer than Whisper tiny.en and Moonshine tiny in full precision on a laptop, including in real noise and reverberation. Measured on a physical board, the firmware runs at 1.97x the duration of the audio on one core, with transcripts identical to an instruction-exact emulator: accurate, but not yet real time (a 4 s command is transcribed about 8.7 s after it ends). The two-core schedule gives wrong results on silicon and is disabled. A streaming variant shows partial text while the speaker talks and delivers the final text 4.1-11.1 s after clips of 3.6-10.6 s (7.2-22.4 s in utterance mode) at 4.86%/11.03% WER; a Spanish model makes 34-60% fewer errors than multilingual Whisper tiny. Code and models are released.