AI summaryⓘ
The authors explore whether large language models (LLMs) can translate whole documents instead of just sentence by sentence, using context from collections of similar texts. They created a system called PAT that uses examples from U.S. English and Latin American Spanish texts to help the model produce translations that better fit Spanish language style and organization. Their tests showed that simple prompts don’t improve much, but adding detailed instructions and example documents can lead to better, though not perfect, translation adjustments. The authors conclude that LLMs show promise for more natural, document-level translation but still need improvement. They also discuss how to design translation systems and evaluate their quality effectively.
large language modelsdocument-level translationretrieval-augmented generationcorpus-informed translationpragmatic auto-translatormachine translationdiscourse organizationrhetorical styleMQM evaluationLatin American Spanish
Abstract
Automatic translation systems, from CAT tools to MT, overwhelmingly treat translation as a sentence-by-sentence act. This paper asks whether LLMs can be moved beyond that paradigm through whole-document, corpus-informed translation. We present PAT (Pragmatic Auto-Translator), a RAG-based system that pairs user-configured specifications with context from a comparable corpus of authentic longform texts in U.S. English and Latin American Spanish, passing retrieved paragraph-, section-, and document-level examples to an LLM for whole-document generation. The goal is draft translation for professional verification: target texts reformulated to fit their Spanish-language context, where discourse organization, rhetorical style, and pragmatic norms differ meaningfully from English. We evaluated six automatic translations of essays on generative AI across three projects using a customized MQM typology, assessed by two trained evaluators working from U.S. English into LATAM and Mexican Spanish. Results show that a limited prompt produced no meaningful reformulation, and specifications and corpus-informed translations at times showed substantial reformulation, though not always to effect. We find that LLMs can be moved toward reformulation and away from the sentence-by-sentence paradigm, though more work is needed to improve the effectiveness of those reformulations. In this paper, we discuss considerations related to automatic translation system design, corpus construction, and translation quality evaluation methodology and results.