Input defenses fail against observation attacks on videollms
Do Input-Level Defenses Transfer to Observation-Level Attacks on VideoLLMs?
Computer Vision and Pattern RecognitionCryptography and Security
Summary
Video Large Language Models (VideoLLMs) help analyze videos in important areas like content moderation but simplify their input by selecting and compressing video frames. The authors find that protections applied directly to these frames often fail to stop attacks that exploit how the model processes all video information together. These attacks hide harmful content from the model even when present in every sampled frame. The study shows that improving safety in VideoLLMs needs new protections across the entire system, not just at the input level.
What this means in practice
- •For video analytics teams: Design video analysis systems that incorporate protection throughout the processing pipeline to better detect harmful content despite adversarial attacks.
- •For content moderation service providers: Improve video moderation safety by deploying defenses beyond frame-level input to counteract attacks hiding harmful content across video observations.
Authors
Bangshuo Zhu, Wei Song, Yuxin Cao, Yuezhong Wu, Zhiquan Liu, Yuekang Li, Jingling Xue
Abstract
Video Large Language Models (VideoLLMs) are increasingly deployed in safety-critical applications such as content moderation and video analytics. To process long videos efficiently, VideoLLMs rely on frame sampling, token compression, and modality fusion, which together form an observation pipeline that reduces the raw video to a compact internal representation. Recent observation-level attacks exploit this pipeline to prevent the model from perceiving harmful content, yet no defense has been explicitly designed for this threat. We introduce DefTEval, a controlled evaluation framework that systematically assesses whether input-level adversarial defenses, which operate on the pixel content of already-sampled frames, can mitigate observation-level attacks. Across five VideoLLMs, eleven representative defenses, and five attack types, we find that input-level defenses offer limited and inconsistent protection, with harmful detection rates frequently near zero. Critically, defenses fail even against attacks that embed harmful signals in every sampled frame, indicating that the bottleneck extends beyond sampling omission to the suppression of signals that do enter the model. Token compression discards localized features, and modality fusion systematically down-weights weakened visual signals. Furthermore, defense effectiveness is dominated by model architecture rather than by the defense method itself, and detection rates vary drastically across content categories, exposing structural weaknesses in temporal reasoning. These findings demonstrate that securing VideoLLMs requires system-level robustness mechanisms spanning sampling-aware coverage guarantees, token-level preservation of safety-relevant features, and modality-balanced fusion.