Learning Simple Test-Time Environments for LLM Web Agents
Computation and LanguageArtificial IntelligenceMachine Learning
Summary
The authors found that large language model (LLM) agents often do well in simple, controlled settings but struggle with complex real-world tasks. To help with this, they created a method called Test-Time Environment Decomposition (TTED) that lets agents break down complicated situations into smaller parts and learn from these parts during actual use, without needing labeled data. Their tests showed that learning from these smaller parts improves the agent's overall performance and ability to handle complex tasks better. The authors suggest this approach is important for making LLM agents work reliably in realistic environments.
Authors
Junxuan Li, Zijun Liu, Ziyi Huang, Peng Li, Yuzhou Liu, Ming Yan, Yang Liu
Abstract
Large language model (LLM) agents have demonstrated remarkable proficiency in manually constructed environments, yet their performance frequently collapses when transitioned to complex real-world settings. Existing research largely attribute this degradation to the compositional generalization gaps in LLMs on combinations of multiple simple, well-structured environments. In this work, we propose that LLM web agents can learn simple environment observations at test time. Specifically, we introduce trial steps for agents to decompose a complex environment observation into sub-modules, and implement a label-free learning method, Test-Time Environment Decomposition (TTED), to adapt agent behaviors with experience during inference. Our empirical evaluations demonstrate the framework's efficacy across both synthetic and realistic benchmarks, showing (1) experience gains acquired within simpler sub-environments can be effectively composed to improve performance in the full one, and (2) test-time training on sub-environments can significantly enhance the compositional generalization of agents in real-world web automation tasks. We also provide key insights in the design of the label-free learning algorithm. As more complex environments are accessed by LLM agents, we believe learning environment decomposition skills at test time will be critical for robust real-world deployment.