Adaptive control improves interaction limits in ai agents with fewer tokens
Elastic Horizon: Discovering the Effective Interaction Frontier in Agentic Reinforcement Learning
Artificial Intelligence
Summary
The paper looks at how AI agents that learn by interacting with environments can perform better by adjusting the number of interactions they have during a task. Instead of just increasing interactions up to a fixed maximum, the authors propose a method called Elastic Horizon that adapts the interaction length based on how much improvement is still happening. They find that beyond a certain point, more interactions don't help much but cost more, so their method stops increasing when it detects this. This leads to better success rates and saves computational resources.
What this means in practice
- •For ai developers: Implement dynamic interaction limits in reinforcement learning agents to improve task success while reducing computation.
- •For game engine programmers: Optimize in-game agent training by automatically adjusting interaction length to match effective learning, cutting wasteful computations.
Authors
Gangyi Zhang, Junjie Meng, Letian Zhang, Wei Wu, Yang Zheng, Dong Wang, Yang Liu, Guanjun Jiang, Chongming Gao
Abstract
Scaling the interaction horizon-the maximum number of environment interactions per episode-improves LLM agents on long-horizon tasks, and curriculum-based methods that progressively expand the horizon outperform fixed-horizon alternatives. However, existing schedules are open-loop: they monotonically increase the horizon until a manually specified maximum, with no mechanism to detect when further expansion stops helping. We propose the effective interaction frontier hypothesis: a dynamic boundary beyond which additional interactions yield diminishing returns while cost grows linearly. We then introduce Elastic Horizon, a closed-loop controller that tracks this boundary via the 90th percentile of successful trajectory lengths. On AppWorld and BFCL, fixed-horizon sweeps reveal clear saturation plateaus; Elastic Horizon stabilizes the horizon inside the saturation band from both under- and over-capacity initializations, attains the best success rates across 7B and 14B backbones, and saves up to 25% of per-step trajectory tokens. Our work shifts the paradigm from how to scale interaction horizons to when to stop scaling.