Router signals improve safety detection in vision language models
Reading, Not Manipulating: Leveraging Router Logits for Multimodal Safety in MoE Vision-Language Models
Computation and Language
Summary
Vision-language AI systems sometimes misunderstand combined pictures and text in unsafe ways. The authors found that instead of changing these AI’s internal parts, simply reading a certain internal signal called router logits can help spot unsafe inputs early. They built a small detector that uses these signals to catch unsafe requests before the AI replies, without needing to change the AI itself. This method worked well on different AI models and safety tests, offering a new way to make AI safer by watching what it ‘naturally’ signals.
What this means in practice
- •For ai product teams: Add a safety detector that flags unsafe multimodal inputs early without changing existing vision-language model code.
- •For cybersecurity teams: Monitor internal model signals to identify potential content safety violations in multimodal AI systems before generation.
Authors
Ziyuan Yang, Wenxuan Ding, Shangbin Feng, Yulia Tsvetkov
Abstract
Vision-language models (VLMs) face compositional safety risks where harmful intent emerges from the interaction between visual and textual inputs. As mixture-of-experts (MoE) VLMs become increasingly common, recent work has explored various safety interventions, including prompting, supervised fine-tuning, and routing-based expert steering. However, these methods show inconsistent improvements across models and evaluation distributions, and the intervention into model behavior or internal states introduce safety-utility tradeoffs by over-refusal. Rather than manipulating internal states to steer model behavior, we instead ask whether routing states can serve as diagnostic signals for multimodal safety. We find that router logits indeed provide highly predictive signals of whether a multimodal input is safe or not. Motivated by this observation, we introduce a lightweight router-logit safety detector that reads out routing signals during prompt prefill and identifies unsafe requests before generation, without modifying model parameters or expert routing. Across Qwen3-VL and Kimi-VL, the proposed detector substantially reduces safety errors on the HoliSafe benchmark and resoundingly generalizes to out-of-distribution safety benchmarks featuring different safety patterns, including MISHard and MM-SafetyBench. The success of the proposed router-logit detector also suggests a broader perspective on model internals: rather than focusing only on manipulating internal components to steer behavior, simply reading naturally emerging signals and linking them to an external safety mechanism can provide a simple, effective, and non-intrusive complement to existing safety interventions.