On-device language model safety relies on few key parameters
How Fragile Is On-Device Language Model Safety? Localizing Safety-Critical Parameters for Sparse Fault Analysis
Artificial IntelligenceComputation and LanguageCryptography and SecurityMachine LearningSoftware Engineering
Summary
Small language models running on devices like phones need to be safe and reliable. The authors studied which parts of a popular language model are most important for keeping it safe. They found that only a tiny fraction of the model’s settings are critical for safety, while most of the model is less sensitive. This means protecting or checking only a few parts could improve safety for devices that have limited resources.
What this means in practice
- •For mobile app developers: Ensure on-device language models remain safe by focusing error checking on a small subset of parameters that critically affect behavior.
- •For embedded systems engineers: Design integrity monitoring tools that selectively protect key model weights to maintain safety under limited hardware resources.
Authors
Muhammad Zeeshan Karamat, Christiana Chamon Garcia
Abstract
As small language models (SLMs) are increasingly deployed on resource-constrained and on-device platforms, including as components of agentic systems, the integrity of locally stored model parameters becomes an important safety concern. We investigate whether safety-sensitive behavior in LLaMA-2-7B-Chat is concentrated within a sparse subset of parameters, creating a reduced fault surface for targeted analysis. We study two complementary localization methods: low-rank safety-associated subspace analysis and parameter-level safety--utility importance filtering. Both approaches reveal highly non-uniform safety sensitivity across the network, with the MLP down_proj consistently emerging as a prominent safety-sensitive component and o_proj providing a smaller contribution. Using parameter-level localization, modifying only 0.19% of model weights in down_proj yields 53% Basic ASR and 56% GCG ASR, while tinyBenchmarks accuracy remains at 51.6% compared with a 52.2% unmodified baseline. These results motivate targeted fault analysis and selective integrity protection for language models deployed in resource-constrained, on-device, and agentic settings.