Ai assistants learn personalized behavior from feedback without retraining
Persona Learning from Feedback for Personalized AI Assistants
Machine LearningArtificial IntelligenceComputation and Language
Summary
Personalizing AI assistants to fit different users or tasks can be tricky because it’s not always clear what the AI should change or keep in its responses. The authors introduce a new way to teach AI assistants personalized behaviors using user feedback without needing to retrain the entire system. Their method creates reusable behavior modules that can be selected automatically based on the conversation, improving how well the assistant matches user preferences. Tests show this approach works better than older methods that use fixed personalities or prompting alone.
What this means in practice
- •For ai product developers: Use feedback-trained persona adapters to build personalized AI assistants that adapt behaviors without full model retraining.$Commercial implications: Enables creation of customizable AI assistants with reusable behavior modules for consumer or enterprise applications.
- •For customer support teams: Deploy adaptive assistant personas to better match dialogue styles in customer interactions without needing user-specific fine-tuning.
Authors
Yiran Shen, Yu Xia, Liuyi Yao, Prithviraj Ammanabrolu
Abstract
Personalizing AI assistants requires adapting their behavior to users' preferences and task contexts. Existing work elicits assistant personas through persona prompting or finetuning on persona-conditioned generations. However, persona descriptions alone do not identify what to change or preserve in a particular response, or which behaviors suit the current request. We introduce $\textit{Persona Learning from Feedback (PLF)}$, a framework that learns reusable, preference-grounded assistant personas as independent LoRA adapters on a shared frozen backbone. During training, a frozen self-teacher conditions on persona specifications and response-specific feedback to provide token-level supervision on each adapter's own generated prefixes. A feedback gate focuses corrective updates on responses judged to need improvement. At deployment, a supervised router uses the request or dialogue to select a single adapter or a fixed composition, without user-specific finetuning. We evaluate PLF on single-turn personalization with model-generated critiques and multi-turn tutoring with simulated students' reactions. Across both settings, PLF improves model-judged preference fit over fixed-persona policies and strong baselines including persona prompting and persona distillation, demonstrating how feedback-trained behaviors can be reused for adaptive assistant personalization without degrading general capabilities.