Steering agents improve goal focus in large language model systems
Learning to Steer LLM Agents and Multi-Agent Systems
Machine LearningArtificial IntelligenceMultiagent Systems
Summary
Large language models often work together or use tools to complete complicated tasks, but they can lose track of their main goal partway through, which wastes time and effort. The authors propose a special 'steering agent' that watches these systems as they work and nudges them back on track when they start to drift. This steering agent decides when to make quick corrections and when to use slower, more careful fixes. It learns how to do this by practicing interventions to improve performance and efficiency. Tests show that this steering approach helps both single and multiple language model agents stay focused, even on new kinds of tasks or with different systems.
What this means in practice
- •For ai system developers: Improve reliability and efficiency of complex AI workflows that use multiple LLMs by integrating a steering agent to supervise and correct task execution.
- •For automation engineers: Enhance automated multi-agent systems for long-running tasks by embedding steering agents to monitor and prevent goal drift during operation.
Authors
Leyang Shen, Yaxin Gong, Yang Zhang, Chenxiao Fan, Jiakai Wang, Dong Wang, Yang Liu, Fuli Feng, Chun Kai Ling, Tat-Seng Chua
Abstract
Large language model (LLM)-based agents and multi-agent systems increasingly tackle complex tasks through long-horizon tool use and inter-agent collaboration. However, these systems are prone to deviating from the overall goal as execution unfolds, which wastes execution resources and leads to undesired outcomes, a failure termed goal drift. In this work, we propose a steering agent that supervises execution on the fly and steers the system back on track when drift occurs. The steering agent observes the execution context, deciding when to intervene with a low-cost fast path and how to intervene with a high-cost slow path. To unlock its steering intelligence, the ability to assess goal drift and generate effective interventions, we jointly train the two paths with reinforcement learning, combining a pairwise ranking objective with group-relative policy optimization (GRPO). Experiments on information-seeking search tasks show that the steering agent provides effective supervision for both single-agent and multi-agent systems, improving their performance and efficiency. The learned steering intelligence generalizes to out-of-domain tasks, transfers to unseen executors, and supports controllable intervention frequency without retraining.