Learning to Reason and Use Tools through Unsupervised Fine-Tuning in Task-Oriented Dialog Systems
2026-08-31 • Computation and Language
Computation and Language
AI summaryⓘ
The authors improved dialogue systems by teaching large language models how to use external knowledge better, which helps them give more accurate answers. They created a way to fine-tune these models without needing labeled data by collecting good reasoning examples and improving them step-by-step. Their approach, tested on a task-oriented dialogue dataset, worked better than previous systems—even beating much bigger models that used a different method. They also analyzed where errors happen and how their method works across different types of conversations.
Task-Oriented DialogueLarge Language ModelsReAct frameworkIn-Context LearningUnsupervised Fine-TuningReasoning TrajectoriesSIMMC datasetError AnalysisTool UseCross-Domain Generalization
Authors
Markel Ferro, Oier Lopez de Lacalle
Abstract
Current dialogue systems struggle with dynamic information retrieval, often leading to hallucinations and lower response accuracy. We address this by adapting the ReAct framework for Task-Oriented Dialogue, enabling Large Language Models (LLMs) to access external knowledge and produce factual responses. Mainly, we propose an unsupervised fine-tuning pipeline that harvests reasoning trajectories via in-context learning inference. High-quality samples are filtered using an LLM-based judge to construct a robust training set. This is enhanced by a unsupervised self-improvement loop, where improved checkpoints generate increasingly better trajectories for subsequent fine-tuning iterations. Experiments on the SIMMC dataset demonstrate that ReAct-based systems outperform baselines due to superior reasoning and tool use. Notably, our fine-tuned 8B model surpasses a 70B in-context system. Finally, we present an error analysis, impact of scene complexity, and cross-domain generalization.