Large language models still reason even when asked not to
Thinking Inertia: LLMs Keep Thinking When Told Not To
Computation and Language
Summary
Large language models (LLMs) sometimes keep thinking through problems even when instructed to give only direct answers without showing their reasoning. The authors studied this behavior by creating new ways to measure when models stop or continue reasoning. They found that telling models not to think doesn’t reliably stop them from showing their reasoning, especially on open-ended questions. This ongoing reasoning is called "thinking inertia," and it matters because it can affect how accurately and cleanly models respond.
What this means in practice
- •For chatbot developers: Improve chatbot designs by measuring and controlling unintended reasoning traces in answers to reduce confusion in user interactions.
- •For ai content moderation teams: Use the proposed evaluation metrics to detect when models unintentionally provide reasoning that might reveal sensitive or unintended information.
Authors
Dianqiao Lei, Kevin Qinghong Lin, Pan Lu, Philip Torr, James Zou
Abstract
Large Language Models (LLMs) increasingly ship with explicit "thinking modes", yet their counterpart, "no-thinking", has received far less attention. We study LLMs' no-thinking behavior along two axes. a. How to measure no-thinking? Prior work typically defines no-thinking through proxies such as a disabled thinking mode or the absence of long traces. These proxies are unreliable: disabled thinking modes may still emit reasoning, while long traces may contain filler rather than genuine inference. We instead normalize each response into a pre-answer trace and final answer, and evaluate it at three levels: (i) Empty-Thinking Rate for strict answer-only compliance; (ii) instruction-aware Question-Pre-answer Relevance for similarity between the question and pre-answer trace; and (iii) LLM-as-judge Explicit Inference Rate for visible explicit inference. Together, these metrics distinguish answer-only output, relevant but non-inferential text, and explicit inference. b. How does no-thinking vary across tasks and models? We evaluate six prompting interventions on six LLMs across Boolean, multiple-choice, and open-ended questions. We find that explicit no-think controls cannot reliably eliminate visible inference. Models instead exhibit "Thinking Inertia": explicit inference persists even under strict controls and becomes more prevalent as the answer space opens. Accuracy remains stable on Boolean and multiple-choice tasks, whereas open-ended tasks reveal a trade-off between answer-only compliance and task accuracy. Rewriting the same questions across answer spaces shows that supplying candidate answers makes answer-only responses easier to produce. These findings establish no-thinking as a non-trivial capability: stopping explicit reasoning cannot be assumed from model settings or instructions alone and deserves systematic evaluation alongside reasoning ability.