Bayesian method improves personalized value alignment in large AI models
From Static Personal Values to Contextualized Personalization: Bayesian Personalized Value Alignment for LLMs
Artificial Intelligence
Summary
Large language models like chatbots often need to adjust to what different users care about, but people's priorities change depending on the situation. The authors propose a way to combine a person's fixed values with how important those values are in each situation using Bayesian reasoning, which updates beliefs based on context. Their method, called BaCVA, first figures out how relevant values are for a situation and then personalizes responses more accurately and efficiently. Tests show this approach works better than previous methods.
What this means in practice
- •For ai product developers: Improve user-specific responses by dynamically adjusting AI language model outputs to the user's changing values in different situations.$Commercial implications: Enables development of adaptive AI assistants that better respect individual user values, enhancing user satisfaction and marketability.
- •For content moderation teams: Customize moderation tools to align with varied user value priorities depending on the context of conversations to improve fairness and relevance.
Authors
Hanze Guo, Aixuan Song, Jing Yao, Xiangxu Zhang, Xiaoyuan Yi, Xing Xie, Xiao Zhou
Abstract
Personalized value alignment has become increasingly important as large language models (LLMs) are expected to accommodate diverse user preferences. However, existing methods typically align model outputs with a static value profile across prompts, overlooking that the salience of value dimensions varies substantially across contexts. Inspired by Lewin's Field Theory, which views human behavior as jointly shaped by personal dispositions and situational constraints, we model personal values as priors and context-dependent preferences as posteriors. We propose BaCVA, an inference-time Bayesian Context-aware personalized Value Alignment method that approximates posterior personalized preferences by integrating static personal values with scenario-specific value salience. BaCVA first estimates contextual value salience from generally normative responses, and then employs a dual-view personalization module to infer posterior preferences from complementary personal-value and scenario-driven perspectives. This Bayesian formulation enables more accurate and adaptive personalized value alignment while improving data efficiency via prior values. Extensive experiments on benchmarks demonstrate its superiority over strong baselines.