When Words Are Safe But Actions Kill: Probing Physical Danger Beyond Text Safety in Hidden-State Risk Space
2026-07-16 • Artificial Intelligence
Artificial IntelligenceCryptography and Security
AI summaryⓘ
The authors explore whether the danger of language instructions causing physical harm is the same as general unsafe content in text. They find that physical danger and content danger are represented differently inside large language models. Using this insight, they create PRISM, a method that successfully detects physical safety risks more accurately than existing approaches. They also introduce a new test set without obvious unsafe words to prove PRISM identifies real physical risks, not just suspicious language. Their work shows it is possible to check for physical safety issues at the model's internal representation level, beyond normal text filtering.
large language modelsphysical dangercontent dangerhidden-state analysislogistic probesafety detectionPRISMSafeAgentBenchPhysicalSafetyBench-1Ktext moderation
Authors
Weimeng Wang, Ziqiang Wang, Zihang Zhan, Chuanpu Fu, Qi Li, Ke Xu
Abstract
Large language models (LLMs) increasingly serve as high-level planners for embodied agents, where linguistically benign instructions can become unsafe once grounded in the physical world. We study whether this physically grounded danger is the same safety problem as ordinary text-level content danger. Through hidden-state direction analysis and random-split null tests, we show that content danger (CD) and physical danger (PD) form separable signals in LLM representations across Qwen2.5-3B/7B/14B/32B, Phi-3.5 and SmolLM2. Building on the CD/PD separability, we propose PRISM, a single-layer L2-regularized logistic probe over full hidden states. PRISM achieves 86.2--87.7\% accuracy on SafeAgentBench with 11.7--13.7\% FPR, while same-scale LLM judges over-block safe tasks at 24.7--39.0\% FPR. We further introduce PhysicalSafetyBench-1K (PSB-1K), a contrastive benchmark of 1{,}000 physical-risk pairs without direct harm keywords, to test whether methods detect physically grounded danger rather than explicit unsafe wording. On PSB-1K, PRISM reaches 99.6\% accuracy and 0.7\% FPR, whereas a Qwen2.5-3B judge rejects 67.8\% of safe tasks. PRISM also replicates on SafeText and EARBench, supporting hidden-state probing as a representation-level method for physical safety beyond text moderation.