LLM agents struggle to adapt and explore in changing environments
AdaExploreBench: Benchmarking LLM Agents for Proactive Exploration and Test-time Adaptation in Evolving Environments
Computation and LanguageArtificial Intelligence
Summary
Large language model agents often have to work in places they don’t fully understand and that can change over time. The authors created a new test called AdaExploreBench to see how well these agents can figure out hidden rules in these environments and adapt when the rules change without warning. They found that even the best models still have a hard time exploring effectively and realizing when their previous knowledge is outdated. This means current AI agents can use information well once they have it but struggle to find the right information or notice when it becomes wrong.
What this means in practice
- •For autonomous robotics teams: Improve robotic agents’ ability to explore and adapt autonomously in real-world changing conditions without explicit updates.
- •For software development teams: Develop AI assistants that better detect and adapt to software environment changes during proactive testing phases.
Authors
Rui Wang, Zishen Lai, Jiayu Liu, Cheng Qian, Jeonghwan Kim, Heng Wang, Xuejun Zhang, Jiateng Liu, Bingxuan Li, Shujin Wu, Qing Zong, Tianshi Zheng, Chunyang Li, Qihan Lin, Chen Li, Dadi Guo, Xiusi Chen, Yangqiu Song
Abstract
Large Language Model (LLM) agents are increasingly deployed in unfamiliar and evolving environments, requiring them to proactively explore, understand, and adapt to environmental dynamics. However, existing benchmarks rarely test whether agents can proactively discover how an environment works and adapt as it evolves. To address this gap, we introduce AdaExploreBench, a benchmark for proactive exploration and test-time adaptation in evolving environments. By instantiating environmental dynamics as hidden rules, AdaExploreBench requires agents to actively acquire evidence and infer the rules. Under implicit rule drift, the latent rule changes without announcement, requiring agents to detect from degraded performance that prior knowledge is stale and re-explore. Experiments on eight leading LLMs show that proactive and adaptive exploration remain challenging: GPT-6 Sol achieves only 61.9% calibrated accuracy, while providing sufficient evidence or removing rule drift improves average performance significantly by close to 10%. These results reveal a critical bottleneck in current LLM agents: they can often reason effectively from informative evidence once it is available, but struggle to actively acquire the right evidence and recognize when previously acquired evidence is no longer valid.