LLMs learn when and how to ask questions to clarify confusion
When and How Should an Agent Clarify? CIGAsk: Teaching LLMs to Clarify via Counterfactual Information Gain
Artificial Intelligence
Summary
Large language models often answer poorly when instructions are unclear because they don’t ask questions to clarify what is meant. The authors teach models not only when to ask for clarification but also how to ask effective, specific questions that help get the needed information. They use a training method that rewards both asking at the right times and asking useful questions. Their trained models perform better at clarifying ambiguous queries across different types of tasks without losing accuracy on normal questions.
What this means in practice
- •For chatbot developers: Improve interactive assistants by teaching them to know when and how to ask questions to resolve ambiguous user queries effectively.
- •For customer support teams: Make automated support systems better at requesting clarifications that lead to more accurate and relevant responses to customer questions.
Authors
Yunxiang Li, Xixin Wu, Helen Meng
Abstract
Instruction-tuned LLMs faced with underspecified queries often commit to a single interpretation rather than ask for clarification, producing confidently wrong answers. In our experiments, prompting alone is insufficient: models either ask for clarification on every query or ask vague questions that fail to recover the missing information. Addressing this failure requires learning two coupled skills: when to ask rather than answer and how to ask a question that recovers the disambiguating information. Existing recipes either address only one of these skills or require a separately trained critic. We propose CIGAsk, an RL recipe that teaches both skills through two complementary reward signals within a multi-turn GRPO loop. Counterfactual Information Gain (CIG) compares the gold-answer log-likelihood under a frozen reference model with and without the user response, providing per-turn credit that guides how to ask. The Asymmetric Ambiguity Bonus assigns a signed reward at the terminal token based on the gold ambiguity label, guiding when to ask. Across three clarification benchmarks spanning table, passage, and open-domain QA, CIGAsk-7B outperforms the strongest external baseline despite using a smaller backbone. It also transfers across datasets without per-dataset tuning while preserving single-turn QA performance on out-of-distribution benchmarks.