Fine-grained steering enhances personalized large language model responses
FleetPilot: Fine-Grained Steering Empowers Real-World LLM Personalization
Artificial Intelligence
Summary
Large language models (LLMs) can be better personalized to individual users by adapting to their changing preferences. The authors developed FleetPilot, a method that uses detailed signals from user interactions to guide the model’s responses in real time. Unlike previous methods that need heavy training or generate many alternatives, FleetPilot adjusts the model on the fly using small preference signals, which helps especially when user data is limited or changing. Tests show that FleetPilot improves performance without slowing down the system or requiring costly updates.
What this means in practice
- •For chatbot developers: Customize chatbot responses quickly and efficiently to match individual user preferences with minimal data and without retraining models.
- •For digital assistant teams: Improve assistant functions such as web calls by dynamically adapting responses based on recent user interactions for better relevance.
Authors
Kounianhua Du, Jianxing Liu, Kangning Zhang, Wenxiang Jiao, Yuan Lu, Jiarui Jin, Jianghao Lin, Weiwen Liu, Yong Yu, Weinan Zhang
Abstract
With the rapid evolution of large language models (LLMs), there is an increasing demand for effective personalization techniques that can adapt model behavior to individual user preferences. Personalized LLMs must infer a user's query-dependent preferences from limited, heterogeneous, and evolving interaction histories. Existing approaches rely primarily on in-context personalization, parametric adaptation, or personalized reward modeling, but respectively incur recurring costs in context management, model optimization and deployment, or multi-candidate generation and scoring. These limitations become particularly restrictive when user observations are sparse and the preference evidence relevant to each query varies across contexts and over time. To address these challenges, we propose \modelname, a fine-grained and instance-tailored steering framework that dynamically generates sample-level interference vectors from user data and injects them into the model's forward pass for personalized adaptation. Our approach introduces: match contrast that produces user-relevant signals, fine-grained steering component that captures nuanced signals by hooking activations from attention and MLP layers, and query-conditioned aggregation module that synthesizes these signals into contextually relevant enhancements. Extensive experiments across diverse scenarios, including short-to-long text generation, and web function calling, validate the effectiveness and compatibility of \modelname. Results show that our method consistently brings significant performance gains, and show extreme superiority in fast-shifting and data-sparse environments. Additionally, \modelname rids us of training cycles and incurs light overhead of storage or inference latency. These make \modelname a strong and real-world deployable method. Implementation is available \footnote{https://anonymous.4open.science/r/FleetPilot}.