HAS: Highlight-guided Attention Steering for Multimodal LLM Video Summarization
2026-07-20 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors address video summarization by improving how key frames are selected in videos. Instead of picking only a few important frames, they propose a method called HAS that highlights important parts but still looks at the entire video continuously. HAS guides the AI to pay more attention to highlighted frames while not ignoring other parts, helping it understand the video better. Their tests show that this approach leads to stronger video summarization results.
Video SummarizationMultimodal Large Language ModelsKey Frame SelectionHighlight-guided AttentionAttention SteeringVideo UnderstandingFrame-level Highlight DistributionAI Video AnalysisContinuous Video Input
Authors
Rui Chu, Yingjie Lao
Abstract
Video understanding has become more and more important with the growth of Artificial Intelligence (AI) for video generation. Recently, Multimodal Large Language Model(M-LLM) has shown its capability in video understanding. Video summarization, a specific domain of video understanding, has proven its importance for efficient navigation and retrieval. Both video understanding and video summarization require a good selection of key frames in a video. Current video summarization methods heavily focus on the selected key frames and correlated segment captions. However, existing approaches overlook the perspective of treating the importance of the frames globally. We argue that using discrete selected frames for summarization will not only reduce the understanding coherence, but also lost important information in the video, as well as wasting the original capacity of the MLLMs. In this paper, we propose HAS, a Highlight-guided Attention Steering method for video summarization. We consider a challenging but practical setting where the video given to MLLMs for summarize should be continuous but with highlight guidance. HAS mainly consists of two parts: The first part is to find a continuous frame-level highlight distribution for the video globally. The second part is to apply the highlight distribution as an attention steering vector for the MLLM, targeting a better understanding of the video, and thus during the model inference time, putting more attention on the highlighted frames, while avoiding lost entire information on less highlighted frames through putting less attention instead of forgetting them. We evaluated HAS on a variety of benchmarks, and it has shown convincing performance in video summarization.