Language agents improve decisions by linking actions to outcomes
When History Fails to Become Experience: Action Calibration in Language Agents
Computation and LanguageArtificial Intelligence
Summary
Sometimes, language-based AI agents don't use their past actions well to make better choices. The authors found that these agents improve if they clearly label how each action leads to a result. They taught an agent to rethink and pick important past experiences to help with next steps. This approach led to better task success because the agent understood the consequences of its actions better.
What this means in practice
- •For ai product developers: Build smarter interactive assistants that better learn from past task attempts by linking actions directly to their results.$Commercial implications: This method enables more reliable improvement in conversational AI products by improving learning from previous interactions.
- •For robotic system engineers: Improve robot decision-making by explicitly associating actions with environmental feedback to avoid repeating ineffective steps.
Authors
Jingyu Liu, Zhiwen Wang, Yuxin Jing, Huanyu Zhou, Yong Liu
Abstract
Language agents should draw on prior attempts and environmental feedback to improve subsequent decisions within the same task. However, providing additional interaction history can sometimes reduce task success, suggesting that agents do not consistently use this information effectively. To investigate this limitation, we examine how agents use history. We find that history improves task completion overall, yet much of this benefit persists even when past actions are shuffled. Disrupting the correspondence between actions and observations causes only a modest decline in task success. We therefore hypothesize that agents do not reliably connect past actions with their outcomes when deciding how to proceed. To test this hypothesis, we explicitly label each returned observation as the outcome of the preceding action. This simple annotation improves task success and reduces next-action repetition without introducing new environmental information. Building on this insight, we introduce a learned calibrator that explicitly reassesses past actions and selectively records experience to guide subsequent decisions, improving task success beyond outcome labeling alone.