Speech translation on edge devices corrects weak output automatically
Quality-Aware Self-Correcting Speech Translation on an Edge Device
Computation and Language
Summary
Translating speech from one language to another usually needs powerful computers or internet connections. This work shows a complete speech-to-speech translation system that runs fully offline on a small Jetson Nano device. The system uses a clever step to check translation quality and tries to fix weak translations without retraining. The authors found one correction approach improves translation quality significantly while saving memory, and their system works live with multiple languages.
What this means in practice
- •For mobile device developers: Create offline speech translation apps that improve translation quality by self-correcting on limited hardware like Jetson Nano.$Commercial implications: Enables selling standalone translation apps for areas with limited internet requiring quality improvements without heavy computation.
- •For embedded system engineers: Deploy efficient speech translation pipelines on small edge devices that adjust translation confidence-driven corrections without retraining models.
Authors
Zubair Ajmal Farooq, Diptesh Kanojia
Abstract
We present a fully offline speech-to-speech translation pipeline that runs on a Jetson Nano (4 GB) and corrects its own weak translations without retraining. A Whisper-tiny ASR feeds an Opus-MT translator; multilingual BERT cosine similarity acts as a Quality Estimation (QE) gate, triggering a secondary-pass correction when confidence falls below a pre-defined threshold $τ$. We compare three correction methods: QE reranking (M1), Minimum Bayes-Risk decoding (M2), and constrained beam search (M3). On 1,012 FLORES-200 sentences (English-Spanish), M2 at $τ=0.90$ produces statistically significant improvements over greedy decoding on BLEU (+0.67, p<0.001), ChrF (+0.51, p<0.001), and COMET (+0.0020 at N=3, p=0.002); M1 yields no significant gains, and M3 is significantly worse than baseline (p>0.99). Our central finding is that QE functions effectively as a gate but poorly as a ranker: removing the QE model from candidate selection (M1$\to$M2) does not hurt quality and frees 680 MB from the critical path. Using a gain-to-edit ratio adapted from the post-editing-effort literature, we further show that smaller candidate pools (N=3) yield more surgical corrections with better semantic adequacy, while larger pools (N=10) maximise lexical reward. We release the system and demonstrate live translation across six language pairs.