HARP improves coordination of AI agents with hidden preferences
Large Language Model Orchestration under Heterogeneous Preferences via Explicit Persona Inference
Computation and Language
Summary
Sometimes AI agents have private likes or goals that they don’t say out loud, making it hard to predict what they will do. The authors develop a new method called HARP, which guesses each agent’s preferences more clearly by keeping numbers about likely preferences and updating them logically. This helps the AI team work better together by fixing early mistakes quickly. They also enhance HARP to keep learning even when actions don’t clearly show preferences. Tests show their method works well across different situations and scales.
What this means in practice
- •For multi-agent system developers: Coordinate autonomous agents whose private goals must be inferred for joint task success using HARP's explicit preference updating.
- •For autonomous robotics teams: Improve cooperation in teams of robots that have different hidden objectives by using HARP to better infer and adapt to each robot’s preferences.
Authors
Shuqing Shi, Ziyan Wang, Milind Tambe, Yali Du
Abstract
LLM orchestration investigates how an orchestrator coordinates a group of autonomous agents to achieve common goals or maximize collective welfare. The agents are typically heterogeneous, each holding a private preference that it pursues but does not reveal. Inferring such hidden preferences from behavior has been a subject of long-standing research in game theory and multi-agent systems. The core challenge lies in maintaining a belief over every agent's preference and updating it from the agents' observed actions. Existing LLM orchestrators carry that belief as prompt text with no explicit update rule. This lets early errors persist and propagate rather than be corrected. We therefore propose \textbf{HARP} (Heterogeneous-preference Agent oRchestration via Preference inference), a novel framework that moves the belief out of the prompt. Specifically, HARP maintains one numeric posterior per agent over a finite set of candidate preferences and updates it in closed form by Bayes' rule. The language model supplies only actions and per-candidate likelihoods, so estimation is decoupled from its reasoning. We prove that HARP attains the same $\tilde O(\sqrt K)$ Bayesian regret as explicit joint inference when the factorization is exact. Furthermore, HARP\textsuperscript{+} augments planning with a bonus for actions that distinguish the candidates, so inference continues even when the optimal action is uninformative. Empirical results on three substrates, ranging from payoffs the preferences fully determine, through payoffs that depend on more than them, to scales where explicit joint inference is infeasible, demonstrate that HARP\textsuperscript{+} is the strongest non-oracle method across the class our theory identifies.