TinyAudio enables text to audio on low resource devices
TinyAudio: Compact and Efficient Text-to-Audio Generation for Low-Resource Deployment
Sound
Summary
Generating audio from text usually needs very large and complex models that are hard to run on simple devices like smartphones. This paper presents TinyAudio, a much smaller and efficient model that can generate high-quality audio from text using far fewer computer resources. The authors created three parts: a text-to-audio transformer, a text encoder aligned with audio features, and an audio decoder. They also made a faster version that can run in real-time on a low-core CPU. Their system shows that good audio generation is possible even on devices with limited memory and processing power.
What this means in practice
- •For mobile app developers: Integrate efficient text-to-audio models for converting text descriptions to sounds on smartphones with limited memory and processing power.
- •For embedded system engineers: Deploy compact audio generation on embedded devices to add audio feedback or description features without requiring cloud access.
Authors
Junxi Liu, Xiquan Li, Wenhao Guan, Yifan Duan, Zhikang Niu, Yanru Huo, Ziyang Ma, Xie Chen
Abstract
Text-to-audio (TTA) generation has advanced rapidly in generation quality and instruction following. However, representative systems often require around a billion parameters, limiting deployment on resource-constrained devices. This paper introduces TinyAudio, a compact flow-matching-based TTA model for low-resource deployment. At its core, TinyAudio uses TA-DiT, a 35M single-stream flow-matching Transformer. TinyAudio also includes TA-CLAP, a 32M audio-aligned text encoder, and TA-VAE, whose 20M decoder reconstructs 44.1 kHz audio from compressed latents. TinyAudio has only 87M parameters in total, over 90% fewer than representative billion-parameter pipelines, and uses 0.48 GB peak GPU memory. TinyAudio achieves competitive generation quality on AudioCaps and TTA-Bench. We further introduce TinyAudio-MF, a MeanFlow-accelerated model that enables real-time generation with a four-core CPU quota. Our results demonstrate a practical quality-footprint trade-off for low-resource TTA deployment.