AlignAtt4LLM: Fast AlignAtt for Decoder-Only LLMs at IWSLT 2026 Simultaneous Speech Translation Task
2026-06-02 • Computation and Language
Computation and LanguageArtificial Intelligence
AI summaryⓘ
The authors present AlignAtt4LLM, a system for translating spoken English into German, Italian, and Chinese in real-time. It works by first converting speech to text and then translating that text incrementally using a special attention-based method adapted for a type of language model that usually lacks certain attention features. They introduce techniques like marking source text spans and selecting specific attention parts to make this work. Their system performs better than baseline models for German and Italian in tests, and although results are mixed for Chinese, the method can be applied to other translation models as well.
simultaneous speech translationforced alignmentdecoder-only language modelcross-attentionattention headsquery/key captureincremental translationlow-latencycascade systemprompt engineering
Authors
Quentin Fuxa, Dominik Macháček
Abstract
We describe AlignAtt4LLM, an IWSLT 2026 simultaneous speech translation system for English to German, Italian, and Chinese. The system is a synchronous cascade: Qwen3-ASR with forced alignment produces an incrementally updated source transcript, and Gemma-4 E4B-it translates that prefix under an MT-side AlignAtt policy. To our knowledge, this is the first application of AlignAtt to a decoder-only LLM, where the encoder-decoder cross-attention used by earlier AlignAtt systems is absent. We recover a usable policy by proposing (1) an explicit source span in the prompt, (2) offline selection of translation-specific alignment heads, (3) selective qk-fast replay of the draft-to-source attention block, and (4) runtime query/key capture that preserves model outputs bit-identically. On the IWSLT 2026 development set, AlignAtt4LLM outperforms the supplied baselines for the European target languages, English to German and English to Italian, in both the low-latency regime around 2 seconds and the high-latency regime below 4 seconds CU-LongYAAL. Results for English to Chinese are more mixed, but the method is not tied to Gemma-4: because AlignAtt4LLM only requires a deterministic prompt layout, calibrated attention heads, and query/key capture, the same policy can be reapplied to stronger translation-focused decoder-only MT backbones for non-European target languages.