New method improves low-rank adaptation of large language model agents
Behavior Quotient Learning for Low-Rank Adaptation of LLM Agents
Machine LearningArtificial Intelligence
Summary
Large language model agents need to perform many different tasks by interacting with their environment. To make these agents better, people often train multiple small adapters, which takes up a lot of storage and slows down the system. The authors found a way to train just one small adapter by carefully organizing and compressing updates from different tasks. Their method helps keep the model’s decisions accurate while using less memory and reducing slowdowns during use.
What this means in practice
- •For ai software developers: Create more efficient AI agents that handle diverse tasks with a single low-rank adapter using less storage and faster inference.
- •For embedded system engineers: Deploy large language model agents on devices with limited memory by compressing multiple task behaviors into one adapter while preserving performance.
Authors
Pengyang Zhou, Xiaobin Tu, Zhengxi Liu, Rongkun Xue, Haochen Li, Miancan Liu, Ziyuan Chen, Yinggui Wang, Jinkui Ren, Xiantao Zhang
Abstract
LLM-based agents rely on heterogeneous interaction capabilities to accomplish complex tasks. Existing approaches often distribute these capabilities across multiple LoRA adapters, which increases adapter storage requirements and introduces routing overhead during inference. A single LoRA avoids this overhead, but learning from diverse agent trajectories under a fixed rank budget presents two challenges. First, trajectories with different interaction traces and parameter gradients can induce equivalent changes in decision distributions, causing repeated updates to overemphasize redundant behavioral changes. Second, an aggregated update may exceed the rank budget of the adapter, and approximating it in weight space can distort the decision changes that it is intended to produce. We propose BQ-LoRA, a low-rank adaptation framework that organizes trajectory updates through a local behavior quotient manifold. It contains two modules, i.e., behavior quotient balancing (BQB) and decision preserving compression (DPC). BQB constructs the quotient manifold from decision distributions and reweights trajectory update directions according to their local density in the quotient tangent space. DPC projects the balanced gradient onto the intrinsic fixed rank tangent space and refactorizes the resulting target by jointly controlling effective weight error and distortion of decision distributions. Experiments on AppWorld and BrowseComp-Plus compare BQ-LoRA with standard LoRA and recent low-rank adaptation methods, while separate ablations evaluate the complementary contributions of both components.