Multilingual models improve target language text with less tuning
Target-Language Generation in Multilingual Models: Activation Steering and Optimal Control
Computation and Language
Summary
It is hard to make multilingual language models write clearly in one specific language. The authors developed a new way to control these models so the text they generate sticks closely to the desired language. They also created a system to check how well the text follows the language and makes sense. Their method works as well as a popular older method but needs less trial-and-error tuning to get right.
What this means in practice
- •For multilingual ai developers: Generate text reliably in a chosen language with minimal tuning for multilingual applications.
- •For translation platform engineers: Improve the language consistency of machine-generated translations by steering models toward target languages.$Commercial implications: Supports building more accurate and fluent multilingual translation tools that can be marketed to global clients.
Authors
James A. Michaelov, Carmen Amo Alonso, Tyler A. Chang, Roger P. Levy
Abstract
Ensuring that multilingual language models generate coherent text in a specific target language is a major issue in multilingual language modeling. We develop an optimal control method for target-language text generation as well as a framework for evaluating the quality of generated text in terms of language adherence, linguistic coherence, and semantic coherence. We find that the proposed method performs at least as well as the prominent difference-in-means activation steering method for the majority of models tested, with substantially less hyperparameter tuning required.