Personalized routing cuts llm cost while meeting user preferences
Chance-Constrained Personalized LLM Routing with Online Preference Learning
Machine Learning
Summary
This work tackles the problem of choosing between different AI language models when responding to a user. The authors propose a way to learn what each user prefers by observing their feedback, even though preferences are initially unknown. Their method balances the cost of using these models with the need to keep users satisfied. They provide theoretical guarantees that the system gets better over time and maintains reliable quality. This approach can also be used for selecting other types of models beyond language systems.
What this means in practice
- •For cloud service engineers: Optimize cloud AI infrastructure by routing requests to models balancing cost and user satisfaction without prior knowledge of preferences.
- •For automated decision system builders: Improve real-time model selection in systems that must maintain reliability under unknown user needs by learning preferences from feedback.
Authors
Ameya Anjarlekar, Rasoul Etesami, R Srikant
Abstract
We study personalized LLM routing, where a router selects among heterogeneous models with different costs and capabilities. The router must minimize cumulative inference cost while ensuring reliable realized response quality, even though user preferences are unknown and must be learned from feedback during deployment. We formulate this problem as a constrained contextual bandit and develop an algorithm that learns preferences while balancing cost and quality. Under a logistic preference model and a strict-feasibility condition, we establish sublinear expected cost regret relative to an optimal stationary randomized contextual oracle and a high-probability guarantee on realized cumulative satisfaction. The framework extends beyond LLM routing to broader online model-selection problems with reliability requirements.