Multi-turn agents refined to cut cost and boost accuracy
Dependency-Aware Trajectory Refinement for Efficient Multi-Turn Agent Fine-Tuning
Computation and Language
Summary
When computers answer questions in multiple steps, some steps are unnecessary and waste time and resources. The authors suggest looking at these steps like a map that shows which ones really matter for the final answer. By teaching computers only with the important steps, they become better and faster at answering questions. This method was tested on several question-answering tasks and improved accuracy while reducing the time and computing power needed.
What this means in practice
- •For ai platform engineers: Reduce computational costs and improve accuracy when fine-tuning multi-turn agent models by using dependency-aware trajectory refinement.
- •For customer support automation teams: Develop more efficient multi-step dialogue agents that provide reliable answers faster by removing redundant reasoning steps during training.
Authors
Zhuo Chen, Zhen Zhang, Xinyu Wang, Kewei Tu
Abstract
Multi-turn agent trajectories often contain redundant rounds (failed tool calls, parallel sub-queries, verification-only steps) that inflate both training and inference cost. We propose viewing each trajectory as a \emph{round-level dependency DAG} that exposes which rounds are globally load-bearing for the final answer, and fine-tune agents on trajectories refined through this DAG. Given an LLM-annotated DAG, these edits are deterministic and interpretable, with optional rephrasing. Models trained on these refined trajectories consistently outperform those trained on the original trajectories at lower inference cost. Specifically, across four multi-modal QA benchmarks, our refinements improve downstream accuracy by up to $1.7$\,pp over vanilla SFT (and $5.7$\,pp over an LLM-deletion baseline) while reducing per-sample inference messages by up to approximately $40\%$ and inference tokens by up to approximately $48\%$, translating to substantial savings in compute and serving cost. Code is available.