Elastic Horizon improves efficiency of AI learning with adaptive limits
Elastic Horizon: Discovering the Effective Interaction Frontier in Agentic Reinforcement Learning
Artificial Intelligence
Summary
Long tasks require AI agents to interact many times with their environment, but using too many interactions can be costly and inefficient. The authors identify that there is a point, called the effective interaction frontier, where extra interactions stop helping much but still add cost. They develop Elastic Horizon, a method that adjusts this interaction limit automatically based on successful task completions. Their approach finds the right balance between too few and too many steps, improving success rates and saving computational resources compared to fixed settings.
reinforcement learninginteraction horizonagentic AIcurriculum learningdynamic controltrajectory lengthsuccess ratecomputational efficiency
Authors
Gangyi Zhang, Junjie Meng, Letian Zhang, Wei Wu, Yang Zheng, Dong Wang, Yang Liu, Guanjun Jiang, Chongming Gao
Abstract
Scaling the interaction horizon-the maximum number of environment interactions per episode-improves LLM agents on long-horizon tasks, and curriculum-based methods that progressively expand the horizon outperform fixed-horizon alternatives. However, existing schedules are open-loop: they monotonically increase the horizon until a manually specified maximum, with no mechanism to detect when further expansion stops helping. We propose the effective interaction frontier hypothesis: a dynamic boundary beyond which additional interactions yield diminishing returns while cost grows linearly. We then introduce Elastic Horizon, a closed-loop controller that tracks this boundary via the 90th percentile of successful trajectory lengths. On AppWorld and BFCL, fixed-horizon sweeps reveal clear saturation plateaus; Elastic Horizon stabilizes the horizon inside the saturation band from both under- and over-capacity initializations, attains the best success rates across 7B and 14B backbones, and saves up to 25% of per-step trajectory tokens. Our work shifts the paradigm from how to scale interaction horizons to when to stop scaling.