Framework improves multi-stage case retrieval for support agents
RAFT: A Stateful Retrieval-Augmented Framework for Troubleshooting Agents
Artificial Intelligence
Summary
Troubleshooting customer support often relies on reusing guidance from past cases, but existing tools treat cases as simple static records. The authors introduce RAFT, a new framework that breaks down cases into multiple steps and retrieves relevant information at these intermediate stages. This approach better matches current problems with similar past situations, improving the ability to find helpful previous solutions. They tested RAFT on synthetic and real-world data, showing it outperforms earlier methods in retrieving useful cases.
What this means in practice
- •For enterprise support teams: Use RAFT to retrieve relevant past troubleshooting cases at each stage to provide more precise guidance to customers.
- •For software maintenance teams: Integrate RAFT to locate intermediate bug-fix states in issue trackers for faster identification of similar problem resolutions.
Authors
Mingxuan Zhang, Xiaowen Wang, Anupma Sharan, Zhengyi Chen, Chenyu Diana Zhang, Shanshan Yang, Chittibabu Pacharu
Abstract
Effective troubleshooting agents in enterprise customer support depend on retrieving actionable guidance from similar historical cases, yet existing retrieval-augmented generation (RAG) systems treat support cases as static documents and overlook their multi-stage, stateful nature. We introduce RAFT (Retrieval-Augmented Framework for Troubleshooting Agents), a stateful RAG framework that abstracts each closed historical case into a directed chain of timeline entries and retrieves at the entry level, surfacing cases whose intermediate states match the active case and returning the parent-case trajectory anchored at the matched state; an optional case-level graph links cases through a configurable similarity representation. We evaluate this retrieval layer directly, which, unlike evaluating a full agent system, requires no production deployment. Because public multi-stage troubleshooting data is extremely rare, we pair a synthetic benchmark built from Microsoft Learn Windows Server documentation with real Apache Jira issues carrying human-created duplicate labels. RAFT improves Case Hit over vanilla RAG and GraphRAG baselines at every stage of case progress, with statistically significant gains over the strongest baseline; the Jira results provide directional evidence that the advantage transfers to real case histories. We release our benchmark, implementation, and the Apache Jira evaluation set.