Debate driven approach improves query to agent matching capabilities
Debate-to-Skill: Capability-Bound Process Supervision for Industrial Query-to-Agent Annotation
Artificial Intelligence
Summary
Matching customer queries to the right software agents often fails when being topically related is confused with being able to actually perform the task. The authors introduce Debate-to-Skill, a method that supervises the decision process behind matching by having structured discussions, decision rules, and automated verification steps. This approach helps better identify which agents truly have the capability to execute requests, especially when the connection is not obvious. The authors tested their method against other approaches and showed it improves the accuracy of these capability-based matches.
What this means in practice
- •For customer support teams: Improve routing of complex or ambiguous queries to capable automated agents by refining capability recognition beyond topical relevance.
- •For enterprise it service managers: Better match service requests with workflow automation tools by focusing on executable capability rather than just related keywords.
Tested on one dataset.
Authors
Shiyu Zhang, Leisheng Cheng, Huifu Li
Abstract
Industrial query-to-agent matching fails when topical relevance is mistaken for executable capability, especially on long-tail and boundary-sensitive requests. We formulate annotation as \emph{capability-bound process supervision} and instantiate it with Debate-to-Skill, which uses reusable decision principles, structured deliberation, verifier-based verdict extraction, and disagreement-driven refinement. On an industrial Query2Agent benchmark, we compare Debate-to-Skill with direct-label supervision, reasoning-SFT, and structural ablations. The results test whether gains come from supervising the capability-critical decision process itself, especially on grey-zone cases where semantic relatedness and executable capability diverge.