Tiny neural codec enables clear voice on low power devices
Narrowband Voice Communication Using Streaming Neural Compression
Sound
Summary
Sending clear voice messages using very little data is hard on small gadgets like microcontrollers. The authors made a simple neural network system called TinyCall that compresses speech neatly so it sounds natural even with tiny bitrates. It runs in real time on small devices like Raspberry Pi and keeps important qualities like speaker identity and expressiveness. They developed tricks to make the system fast and stable, replacing complex parts with simpler ones suitable for limited hardware. Their tests showed clear speech at very low bitrates, proving neural voice communication can work on devices with very limited resources.
What this means in practice
- •For emergency communication teams: Deploy ultra-low bandwidth voice communication tools that work on tiny, low-power devices during disaster scenarios.
- •For iot device developers: Implement bandwidth-efficient voice transmission in smart home or sensor devices with limited computing and power budgets.$Commercial implications: Enables selling compact, affordable IoT gadgets with voice communication over constrained networks where current solutions fail.
Authors
Dahong Luo, Anannya Trehan, Aritrik Ghosh, Nirupam Roy
Abstract
Low-bitrate speech communication on resource-constrained edge devices remains challenging due to stringent computational, memory, and bandwidth constraints. We present TinyCall, a lightweight neural audio codec designed for real-time speech communication on low-power platforms such as the ESP32 microcontroller and Raspberry Pi. The proposed system targets emergency communication and other bandwidth-limited scenarios while preserving speech intelligibility, speaker identity, and vocal expressiveness. To enable efficient deployment, we propose a minimal neural audio codec architecture together with a framework for converting a causally trained codec into a truly streamable codec through pseudo-lookahead decoding and decoder-input caching. We further replace conventional residual vector quantization (RVQ) with Residual Finite Scalar Quantization (RFSQ) to reduce inference complexity on edge processors and employ a progressive three-stage training strategy for stable optimization under latent quantization. An MFCC-based perceptual loss encourages preservation of speaker characteristics, including harmonic structure and vocal timbre. Experimental results demonstrate real-time operation on a Raspberry Pi 3 while achieving intelligible speech reconstruction at bitrates as low as 2.3 kbps. The proposed approach demonstrates that practical neural speech communication is feasible on highly resource-constrained edge devices.