ECHO: Learning Epistemically Adaptive Language Agents with Turn-Level Credit
2026-06-29 • Multiagent Systems
Multiagent Systems
AI summaryⓘ
The authors explore how language agents can better decide what information to gather and when to act based on what they know. They create a new framework called Epistemic Decision Processes (EDPs) to model how an agent's beliefs change as it gathers evidence. Then, they develop a method named ECHO that helps the agent learn from each step instead of just the whole journey, improving its ability to seek useful information efficiently. Their tests show that ECHO outperforms older methods in an evidence-gathering game without needing extra explanations.
Epistemic Decision Processesbelief statemulti-turn information seekingposterior updateBayesian inferencepolicy gradientreinforcement learningepistemic adaptivityinformation gaincalibration
Authors
Abhijnan Nath, Nikhil Krishnaswamy
Abstract
What does it mean for a language agent to be adaptive? Effective multi-turn agents must decide what information to seek, how to use new evidence, and when they are certain enough to act. We introduce Epistemic Decision Processes (EDPs), a belief-state formulation of multi-turn information seeking in which actions produce external observations that update the agent's posterior over a latent task variable. EDPs make epistemic adaptivity explicit: good policies choose actions that are useful under the current belief, not merely those that correlate with eventual success. We prove that belief-agnostic policies can suffer errors that compound exponentially over the horizon, and that aggregate trajectory returns can fail to identify the per-turn Bayesian advantage needed for epistemic credit. We then introduce ECHO (Epistemic Credit for History-Conditioned Optimization), a practical clipped policy-gradient objective that assigns turn-level credit using posterior-sensitive rewards. In the Clue Selector Game, a novel controlled evidence-seeking benchmark, we show that ECHO substantially improves resolution, information gain, and efficiency over trajectory-level GRPO, and matches or exceeds frontier baselines on epistemic metrics such as grounding, recovery, and calibration while producing almost no visible reasoning text.