Papers for

conference interpreting platforms

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Model balances speed and quality in streaming speech translation

S3-S2ST: Semantically Supervised Streaming Speech-to-Speech Translation

Abstract: Streaming speech-to-speech translation requires timely decisions about when partial input is sufficient for accurate translation. We present S3-S2ST, a model that learns to translate and speak from semantically aligned streaming supervision. By converting utterance-level speech pairs into incremental training examples, we teach a learned policy when to commit translations. Its adjustable threshold controls the quality--latency tradeoff on a fixed 640\,ms input grid. S3-S2ST delivers competitive English--Chinese translation on FLEURS and leads the evaluated streaming systems in translation quality on the internal 9-pair test set. On RealSI, adapting the threshold to buffered speech further improves quality over a low fixed threshold with little change in first-audio latency. We plan to release the model and code.

Tue 27 OctSound

The gist

Translating spoken language in real-time is tricky because the system needs to decide when it has heard enough to give a good translation without waiting too long. The authors created a model called S3-S2ST that learns when to start translating and speaking based on the meaning of what’s been said so far. It works by turning whole speech sentences into smaller chunks for training, helping the system decide the best moment to translate. This approach allows adjusting the balance between translation accuracy and how quickly it responds. The model performs well on English-Chinese translation tasks and shows improvements in translation quality with smart adjustments.

Open → 2610.36777v1