Deep research agents struggle with vague user queries
How Specific Do I Need to Be? Effects of User Query Underspecification on Deep Research Agents
Computation and Language
Summary
Deep research agents are computer tools that answer complex questions by looking up and interpreting information. The authors found that when questions are not specific enough, these agents give lower-quality answers even though they still use real sources. This happens because the agents search wider and rely more on what they learned before rather than the user’s exact question. Their work shows these agents need ways to detect unclear questions and ask users for more details to improve answers.
What this means in practice
- •For product teams building ai assistants: Improve question handling by detecting vague user queries and prompting for clarification to deliver better research answers.
- •For enterprise knowledge management teams: Design search systems that adjust query strategies when initial queries lack detail to find more relevant evidence for decision support.
Authors
Yi Wen, Xiangjue Dong, Bingxuan Li, Hangxiao Zhu, Leo Zhicheng Liu, Yu Zhang, James Caverlee
Abstract
Deep research agents (DRAs) conduct long-horizon reasoning, search, and literature synthesis to provide long-form answers to complex, domain-specific questions. These agents are typically trained and evaluated on fully specified user queries. However, real-world user queries can often be underspecified. We introduce UNDERSPEC-DR, a framework for systematically evaluating how underspecified queries affect agents' final answer quality and intermediate search behavior. Across three DRAs we evaluated, underspecification substantially reduces how well DRA's answers fulfill users' information needs, even though they remain grounded in cited evidence. Our analysis of intermediate search behavior suggests a possible explanation: DRAs conduct broader searches that draw less on the original user query but rely more on their learned prior during training, which can be misaligned with users' intent. Our findings reveal the vulnerability of current DRAs to underspecified user queries and inform future efforts to build DRAs that proactively recognize when additional user input is needed to reliably complete deep research tasks.