Summary
Large language model (LLM) agents that use external tools sometimes get answers wrong even when the tool gives the correct result. This happens when agents don’t read the full tool output in context but instead retrieve fragments or summaries, leading to what the authors call evidence substitution, where the agent uses less accurate evidence from memory or smaller files. The authors tested this by having AI agents use a complex geochemistry simulation tool with very long reports and found that certain ways of managing context cause accuracy losses, while others do not. They also found that naming or including authoritative output improves accuracy, highlighting the importance of how AI systems handle long tool outputs.
What this means in practice
- •For ai system engineers: Optimize context and retrieval methods in AI agents to minimize accuracy loss from handling long tool outputs.
- •For scientific software developers: Improve integration and output management so AI agents can better use large simulation reports for precise automated answers.
Authors
Ke Zhang, Sahchit Chundur, Jerry Peprah Owusu, Mohammad Javad Abdolhosseini Qomi, Maziar Raissi
Abstract
LLM agents that compute the right answer can still answer wrongly. Agent harnesses increasingly keep long tool outputs out of the context window, saving them to files that the agent searches or reads on demand. We ask where this costs accuracy with a real scientific tool: PHREEQC, a widely used geochemistry simulator from the USGS, whose reports reach 846.8k characters and whose determinism lets every printed value be traced. Agents use it to answer 158 execution-verified multiple-choice items. When agents read the simulator's report through a section index rather than in context, GPT-5.1 and GPT-5.2 lose 11.3 and 7.9 points of accuracy, while Claude Opus 4.6 and Claude Sonnet 4.6 show no detectable cost. Re-running all 11,007 logged simulator calls shows that both GPT models compute the keyed value about as often as before; what changes is the evidence they answer from. Under retrieval they often skip the report, answer from their own higher-precision output files, and misround to a neighboring option, a failure we call evidence substitution. With the simulation held fixed, retrieval still costs GPT-5.2 9.4 points. For GPT-5.4, more frequent correct simulations mask an evidence-use loss of about ten points, leaving a net change indistinguishable from zero. Naming the authoritative output raises accuracy by 6.3-7.9 points over plain retrieval for the three GPT models; for GPT-5.1 and GPT-5.2, inlining outputs that fit works as well, while asking for correct rounding does not. Context-management choices in agent harnesses should be judged by where accuracy is lost, not only by final accuracy.