Video large language models need new defenses for hidden attacks
Do Input-Level Defenses Transfer to Observation-Level Attacks on VideoLLMs?
Computer Vision and Pattern RecognitionCryptography and Security
Summary
Video large language models (VideoLLMs) are used to analyze long videos but rely on reducing video data into simpler forms to work efficiently. The authors found that current defenses applied to the input frames do not effectively stop attacks that manipulate how the model sees harmful content. These attacks take advantage of the model’s way of compressing and combining information, which can hide important signals about dangerous content. The study shows that improving security for these models requires system-wide strategies that consider how videos are sampled, how features are preserved, and how different types of information are combined.
Video Large Language ModelsObservation-level attacksInput-level defensesFrame samplingToken compressionModality fusionAdversarial attacksRobustnessVideo analyticsContent moderation
Authors
Bangshuo Zhu, Wei Song, Yuxin Cao, Yuezhong Wu, Zhiquan Liu, Yuekang Li, Jingling Xue
Abstract
Video Large Language Models (VideoLLMs) are increasingly deployed in safety-critical applications such as content moderation and video analytics. To process long videos efficiently, VideoLLMs rely on frame sampling, token compression, and modality fusion, which together form an observation pipeline that reduces the raw video to a compact internal representation. Recent observation-level attacks exploit this pipeline to prevent the model from perceiving harmful content, yet no defense has been explicitly designed for this threat. We introduce DefTEval, a controlled evaluation framework that systematically assesses whether input-level adversarial defenses, which operate on the pixel content of already-sampled frames, can mitigate observation-level attacks. Across five VideoLLMs, eleven representative defenses, and five attack types, we find that input-level defenses offer limited and inconsistent protection, with harmful detection rates frequently near zero. Critically, defenses fail even against attacks that embed harmful signals in every sampled frame, indicating that the bottleneck extends beyond sampling omission to the suppression of signals that do enter the model. Token compression discards localized features, and modality fusion systematically down-weights weakened visual signals. Furthermore, defense effectiveness is dominated by model architecture rather than by the defense method itself, and detection rates vary drastically across content categories, exposing structural weaknesses in temporal reasoning. These findings demonstrate that securing VideoLLMs requires system-level robustness mechanisms spanning sampling-aware coverage guarantees, token-level preservation of safety-relevant features, and modality-balanced fusion.