Hardware method improves safety and efficiency of small language models

MOMAT: Mixture of Multiple Atlases for Low-Power Jailbreak Defense of Quantized LLMs

Cryptography and SecurityArtificial Intelligence

Summary

Quantized large language models (qLLMs), which run on low-power devices, are vulnerable to tricks that make them behave badly. The authors propose MOMAT, a system that uses multiple groups called atlases to organize safe and unsafe examples, helping the model better recognize harmful inputs. MOMAT uses special low-energy hardware to speed up checking prompts against these atlases, making the safety process both faster and much less energy-intensive. Tests show MOMAT keeps qLLMs safe while saving a lot of power and running much faster than older methods.

What this means in practice

  • For edge device engineers: Enable qLLMs on edge hardware to resist jailbreak attacks while minimizing power use and latency.
  • For ai safety teams: Deploy modular retrieval-based defenses to improve safety robustness of quantized language models without overblocking safe inputs.

Authors

Boyang Li, Bingyu Shen, Weihao Hong, Zhiyuan Jiang, Xinlei Guan, Yan Ma, Miles Q. Li, Yi Sheng, Ruiyang Qin

Abstract

Quantized large language models are increasingly deployed on edge devices for their low latency and energy efficiency. However, model quantization weakens alignment safeguards, leaving qLLMs (quantized large language models) highly vulnerable to jailbreak attacks. To address this challenge, we present MOMAT (Mixture of Multiple Atlases), a hardware-enhanced safety framework that combines structured knowledge retrieval with low-power defense acceleration. Each atlas represents a semantic cluster of harmful or benign sample sets and policy templates, enabling domain-localized Retrieval-Augmented Generation guarding that mitigates the curse of dimensionality and the resulting semantic sparsity problem in large, heterogeneous safety databases. MOMAT retrieves top-$k$ similarity features from all atlases for each prompt and evaluates them using a lightweight MoE (Mixture of Experts) detector, while a CiM (Compute-in-Memory)-accelerated similarity engine performs fast, low-power atlas-local retrieval. MOMAT's CiM-based retrieval accelerates a 100-query batch from 15,052.44 ms to 3,207.21 ns (a $4.69 \times 10^6\times$ speedup) and reduces energy from $8.1 \times 10^7$ $μ$J to 3.32 $μ$J, yielding an approximately $2.5 \times 10^5\times$ energy reduction over DRAM-based (Raspberry Pi) baselines. Red-team evaluations across standard benchmarks show that MOMAT matches the defense performance of state-of-the-art methods while avoiding benign overkill and providing substantial efficiency gains, demonstrating that CiM-based modular defenses can make edge-deployed qLLMs both safer and more energy-efficient. We will release the full 223.2k-sample dataset to foster future research.