Structured memory improves small language model tool use reliability
Learning from Failures: Heterogeneous Graph Memory for Small Language Model Tool-Using Agents
Machine Learning
Summary
Small and medium-sized language models can use tools to accomplish tasks, but they often make mistakes that cause problems. The authors designed a system called FRESH that helps these models remember past successes and failures in a way that keeps track of how tasks and actions relate to each other. This structured memory helps the models avoid repeating errors and make safer decisions. Tests show that models using FRESH perform better on complex tasks and use tools more reliably.
What this means in practice
- •For software developers: Enhance local AI assistants to manage tools more reliably by using structured memory of past successes and failures.
- •For cloud service engineers: Deploy cost-effective small language model agents that reduce costly execution errors in long and stateful workflows.
Authors
Jiaxing Li, Lei Song, Rui Dong, Youyong Kong
Abstract
Small and medium-sized language models offer cost-effective executors for tool-using agents, making them attractive for local and large-scale deployment. However, in long-horizon and stateful environments, they often make structural errors such as missing required observations, performing premature writes, repeating failed calls, and violating action preconditions. These errors can lead to incorrect state updates, policy violations, and costly or irreversible consequences, making reliable tool execution a critical deployment challenge. Existing fine-tuning approaches require substantial data and computation, while flat memory may retrieve failed actions without preserving their causal context or safety conditions. In this paper, we propose FRESH, a Failure-aware Retrieval framework over Experience-Structured Heterogeneous graphs, which transforms historical successes and failures into structured external experience for tool-using agents. By explicitly modeling the dependencies among tasks, actions, errors, repairs, and execution conditions, FRESH helps frozen language models reuse reliable strategies, avoid recurring failures, and make safer decisions in stateful tool interactions. Experiments on $τ$-Bench and AppWorld with multiple open-source models show that FRESH consistently improves task success and tool-use reliability over no-memory agents and representative memory-based baselines.