Information-directed method improves agent personalization from comparisons

Bridging Exploration and Persona: Information-Directed Dueling for Test-Time Agent Personalization

Artificial IntelligenceComputers and Society

Summary

Personalizing AI responses to different users based on brief interactions can be challenging. The authors study how to pick the best personal style by asking users to compare pairs of responses. They created a new method called ID-TAP that smartly chooses which comparisons to make and learns from similar questions to adapt faster. Tests show ID-TAP works better than other approaches within a limited number of comparisons. This method helps AI adapt to different user preferences quickly and efficiently.

What this means in practice

  • For chatbot developers: Build chatbots that adapt their style to individual users by efficiently learning from user comparisons during interactions.
  • For customer support teams: Improve AI agents' responsiveness to customer preferences by quickly personalizing responses based on lightweight feedback signals.

Authors

Jie Mao, Pingchen Lu, Xiangyi Wang, Hengquan Huang, Shuang Qiu, Min Zhang, Zhongxiang Dai

Abstract

Effective test-time personalization requires LLM agents to adapt to query-dependent user preferences from lightweight interaction. We study persona selection with binary comparisons between candidate responses as the online signal. We present ID-TAP, a scalable, IDS-inspired neural dueling-bandit framework for large discrete persona spaces. ID-TAP combines a query-conditioned neural utility model with a lightweight Bradley-Terry explorer, and transfers comparison evidence across related queries through similarity-weighted warm starts. The neural model shares statistical strength across queries while preserving query-specific scores, and efficient BT updates score hypothetical challengers. Across seven user-conditioned generation benchmarks under reference- and model-based preference simulators, ID-TAP achieves the strongest aggregate performance among the evaluated baselines within a fixed ten-round comparison budget. These results support information-directed challenger selection as an effective design for test-time persona adaptation.