You Shouldn't Have Asked: A Pragmatics-Inspired Taxonomy for Evaluating LLM Refusals
2026-08-31 • Computation and Language
Computation and LanguageHuman-Computer Interaction
AI summaryⓘ
The authors studied how large language models (LLMs) say no to unsafe or harmful requests, since saying no can sometimes upset the person asking. They created a new way to categorize how LLMs refuse based on communication theories and tested 16 models across different harmful topics. They found that models usually refuse clearly and with moral reasons but don't always handle the social aspect of refusing well, sometimes making users feel bad. The authors suggest that it's important to not only check if models refuse harmful things but also to see if they do it in a sensitive and socially aware way.
large language modelsrefusalspragmaticsface-threatening actsinteractional costsafety alignmentharm contextsmoral evaluationsocial accountabilitynon-compliance
Authors
Ruoxuan Li, Pinqiao Wang, Sheng Li, Cameron Robert Jones
Abstract
Refusals are often treated as face-threatening acts in pragmatics because they can challenge the requester's socially claimed self-image. Large language models (LLMs) are increasingly trained to refuse unsafe and inappropriate requests, and these refusals may harm users when models fail to manage this interactional cost properly. While existing work has mainly approached LLM non-compliance as a safety-alignment outcome, it does not provide a way to evaluate whether LLMs refuse appropriately across different harmful contexts. To study this question, we propose (to our knowledge) the first taxonomy of LLM refusals that is grounded in pragmatic theory. Applying this taxonomy to responses from 16 modern LLMs across 14 harm categories, we find that although models differ in how they refuse, their refusals are overall explicit and strongly morally evaluative, with interactional repair occurring mainly through offering or providing safer alternatives instead of interpersonal facework. This pattern is especially consequential in sensitive harm contexts, where overuse of negative framing may make users feel shamed or provoked, undermining the purpose of safe non-compliance. We therefore call for alignment evaluation that considers not only whether models refuse harmful requests, but also whether they refuse in ways that are contextually adaptive and socially accountable for the interactional consequences of saying no.