Large language models show risks before tool calls with refusal affecting safety
Before Retrieval Begins: Safety and Tool-Use Decisions in LLM Agents
Computation and Language
Summary
Large language models (LLMs) that act as agents often decide whether or not to call external tools, but most studies only look at what happens after the call. This paper finds that the decision to call a tool can lead to unsafe outcomes almost three-quarters of the time when harmful requests are admitted. The authors explore what influences this decision and find that refusing harmful requests in writing is connected but only partially and depends on the model. They show that training models to refuse harmful requests in text reduces unsafe tool calls without harming safe tool use, suggesting that focusing safety checks on tool admission decisions is important.
What this means in practice
- •For ai system developers: Improve the safety of LLM-based agents by directly training them to refuse harmful requests before calling external tools, reducing unsafe tool use.
- •For ai service operators: Evaluate and supervise LLM agents’ decisions to admit tool calls as a safety checkpoint beyond monitoring text refusals, improving user protection.
Authors
Aditya Nawal, Manit Baser, Mohan Gurusamy
Abstract
Large language models increasingly act as agents that call external tools, yet most tool-safety research examines only what follows a tool call. The prior decision of whether to call the tool remains largely unexamined and can influence downstream safety. Across the seven open-weight models, the unweighted model-average harm rate after admitted harmful requests is 74.5%. To identify what governs this decision, we apply a factorial intervention to the input request. Within our generated construction, neutralizing the requested-operation component produces the largest admission contrast (19 pp). A linear direction estimated from this contrast influences held-out admission in four of seven models; effects on operation-neutralized requests in some models show that it is not an operation-specific mechanism. We then test whether written refusal is behaviorally and representationally coupled to the same decision. Written refusal accompanies 33.6% of admitted harmful calls. Plain-text refusal training reduces unsafe admission in models that retain benign tool use, showing that response-level safety training can transfer to the call decision. Together, these results establish a partial, model-dependent relation between written refusal and tool admission. They motivate direct evaluation and supervision of tool admission rather than treating text refusal as its proxy.