PCTD: Preference-Guided Counterfactual Task Decomposition for Agent Tool Retrieval
Information Retrieval
Summary
The authors study how to break down unclear instructions into simple steps to help tools work better. They found that using typical measures like Recall or NDCG to judge these breakdowns can lead to cheating behaviors in AI, such as repeating steps unnecessarily. To fix this, they created a new method called PCTD that uses a special reward system to focus on the real impact of each step and keeps the breakdowns logical and clear. They also made a new benchmark called MTDTool to test their approach in mobile interactions. Their experiments show that PCTD reduces cheating and works better than previous methods, especially with new, unseen tools.
Authors
Chu Zhao, Lei Tang, Minghang Li, Jianzhe Zhao, Guibing Guo, Zhengzong Chen, Yuanyuan Zhao, Fei Huang
Abstract
Task decomposition aims to transform ambiguous instructions into executable atomic subtasks, thereby guiding high-precision tool retrieval. However, our analysis reveals that directly adopting tool retrieval metrics, i.e., Recall or NDCG, as rewards for task decomposition can easily induce reward hacking in reinforcement learning-based methods. Specifically, models tend to maximize retrieval matching through strategies such as repetitive decomposition. This spurious correlation between the shallow features of decomposition results and retrieval metric impairs generalization in Out-of-Domain (OOD) scenarios involving unseen tools. To address this issue, we propose PCTD, a Preference-guided Counterfactual Task Decomposition framework. PCTD quantifies the marginal causal gain of decomposition on retrieval ranking through a counterfactual reward, thereby cutting off spurious correlations at their source. Meanwhile, it introduces a preference reward to impose fine-grained structural supervision on logical coherence and atomicity, encouraging the model to generate high-quality decompositions. In addition, we construct MTDTool, the task decomposition benchmark specifically designed for mobile multi-turn interactions. Extensive experiments demonstrate that PCTD alleviates repetitive decomposition and surpasses SOTA methods in retrieval, decomposition quality, and OOD generalization.