Llm agents struggle to know when to act or ask for help
Ask, Relax, or Act? Evaluating Actionable Indeterminacy in LLM Preference Reasoning
Computation and LanguageArtificial Intelligence
Summary
Sometimes AI language models aren’t sure whether to take an action or ask for more information. This paper studies when an AI should act immediately, ask questions if many options are possible, or suggest changing the problem if nothing works. The authors created tests using things like scheduling and apartment choices to see how well models make these decisions. They found models often act when they should wait, and making clear what kind of response is expected helps improve results.
What this means in practice
- •For ai system builders: Improve AI helpers by better deciding when to act or ask clarifying questions in planning tasks like scheduling or resource allocation.
- •For human-computer interaction designers: Design clearer AI feedback systems that provide expected content formats to reduce unnecessary user interruptions during automated decisions.
Authors
Ang Li, Yue Lin, Feifei Kou, Zhan Su, Prayag Tiwari, Wenhao Li, Shuhui Zhu, Hongyuan Zha, Baoxiang Wang
Abstract
An LLM agent can recognize uncertainty yet still choose the wrong next step: asking when action is already justified, or seeking clarification when the constraints must change. We formalize actionable indeterminacy: act when an accepted action is shared across all admissible preferences or objectives, clarify when each possibility is feasible but no action is shared, and propose a minimum-cost permitted constraint repair when the request is infeasible. We construct a solver-grounded benchmark spanning object allocation, meeting scheduling, apartment choice, and stable matching. Matched pairs retain the same source while changing whether intervention is necessary, and evaluation separates decision correctness, matched-pair reliability, and fully correct responses. Our findings reveal a recurring difficulty in recognizing when intervention is unnecessary: models can identify situations requiring clarification or repair yet still intervene when a justified action already exists. Correct decision labels also fail to guarantee usable actions, questions, or repairs. Crucially, response requirements shape not only how decisions are expressed but also which decisions are made. Making the required content explicit substantially improves fully correct responses and can change intervention decisions, even when outputs are already parseable. These findings highlight that reliable agency requires more than recognizing uncertainty: it requires intervening only when necessary and translating the chosen next step into a verifiable response.