Production greek english speech recognizer meets strict quality gates
Building a Production Greek-English Speech Recognizer
Artificial IntelligenceSound
Summary
This paper describes how a team built a working speech recognition system that can understand both Greek and English spoken language. They tested the system with many challenges, like identifying which language is spoken and ignoring non-speech noise, but found it was tough to get everything perfect at once. To improve the Greek part under noisy conditions, they had to show the system a lot of Greek audio examples, which made maintaining English recognition harder. They also used multiple models working together to improve accuracy, especially when multiple people spoke at once. The team shared detailed methods and results but did not release the actual model or training data.
What this means in practice
- •For speech recognition engineers: Build bilingual Greek-English speech recognizers that better handle noisy environments and language mixing using multi-model ensembles and curated training data pipelines.
- •For call center technology teams: Improve transcription accuracy in bilingual Greek-English calls, including overlapping speech scenarios, by adopting multi-model voting systems informed by this work.
Authors
Christos Petrocheilos, Cleopatra Papadopoulou, Chris Porikis, Ioakeim Perros, Ayoub Kirouane, Themistoklis Nikolis
Abstract
We report a multi-month engineering program to build Sophea, a production bilingual Greek-English automatic speech recognition system. We evaluate the system against nine production gates covering Greek and English word error rate, language identification, and hallucinations on non-speech audio. Across twenty-three training iterations and two model architectures, no training-data composition passed all nine gates simultaneously. Meeting the Greek noisy-environment target required about 1,500 steps of dense domain exposure, while preserving English language identification tolerated only about 250 steps, or about 1,250 with a rebalanced mix that reduced Greek accuracy. We describe a six-stage data pipeline in which calibrating an audio-quality filter against in-domain anchors reduced the discarded share of scored Greek audio from 98.7 percent to 10.6 percent. A pre-registered ablation isolated a hallucination defect to one training-data package. A three-model ROVER ensemble increased gate coverage from 4-7 of 9 for individual models to 9 of 9 and reduced overlapping-speech WER from 53.35 percent to 37.87 percent, a 29 percent relative improvement. A separate learned per-clip arbiter over two models is listed as sophea/asr-k1 (preview) on the public Open ASR Leaderboard, with 4.26 percent average WER across eight public English test sets, and reaches 25.88 percent WER on live Greek noisy-environment traffic. We also document five cases in which a measurement tool produced a plausible but incorrect result and seven substantial approaches that were evaluated but not shipped. No model weights or training data are released; we report methodology and quantitative results only.