Language models show consistent moral biases across multiple languages
Geometry of Values: Task Vector Composition for Ethical Preference Alignment in Language Models
Computation and LanguageArtificial Intelligence
Summary
Large language models sometimes struggle to fairly balance moral values like honesty, justice, and autonomy, especially when faced with tough choices. The researchers created a large dataset of moral dilemmas translated into several languages to study how these models perform across languages. They found that some models had strong biases, often favoring honesty over autonomy, but training techniques helped reduce these biases dramatically. They also developed a new method to isolate and control these value preferences within the models, allowing the model’s stance on ethical questions to be adjusted systematically.
What this means in practice
- •For ai developers: Improve language models’ ability to handle ethical dilemmas by reducing bias and aligning moral preferences across multiple languages.
- •For multilingual chatbot teams: Build chatbots that consistently apply ethical reasoning across various languages by using task vector adjustments to tune model values.
Authors
Utkarsh Agarwal, Monojit Choudhury
Abstract
Large Language Models (LLMs) are increasingly deployed in applications that must weigh clashing moral values, yet even strong models exhibit hidden biases and brittle instruction-following across languages. We introduce a 12,000-instance dataset of two-option dilemmas covering pairwise three value conflicts: Honesty vs. Justice, Justice vs. Autonomy, and Autonomy vs. Honesty, along with their translations into Hindi, Arabic, Spanish, and Chinese, to probe cross-lingual behavior. Benchmarking on GPT-5-mini reveals that it consistently favors Honesty over Autonomy across all five languages when no policy is given. The Llama-3.2-1/3B models exhibit strong first-option bias; however, both plain fine-tuning and Direct Preference Optimization fine-tuning effectively remove this bias, increasing accuracy to greater than 98%. In order to decouple the effect of learning correlations in the dataset from abstract values, we propose a task vector transfer based experiment where after computing the task vectors for a direction of value preference we orthogonalize it with respect to the general instruction following vector. Our experiment shows that this method is effective in isolating the direction of the specific value preference that can successfully be used to conduct task arithmetic to obtain a model with the opposite stance.