Teach Multimodal Recommendation Model to See via Personalized Visual Extraction and Adaptive Learning
2026-06-08 • Information Retrieval
Information Retrieval
AI summaryⓘ
The authors studied recommendation systems that use both text and images to suggest items but found that images weren't helping much. They identified that existing methods struggle to learn useful visual information and focus too much on text. To fix this, they created REVEAL, a method that improves how visual features are extracted and balanced with text without changing existing recommendation models. Their tests show REVEAL helps the system pay better attention to important image parts and improves overall recommendations.
multimodal recommendationvisual representation learningtextual featurescross-modal optimizationprompt-guided extractionadaptive learningsequential recommendationpretrained encodersmodality imbalanceattention mechanisms
Authors
Yutong Li, Xinyi Zhang, Ziyi Ye, Daoguo Dong, Yu-gang Jiang
Abstract
Multimodal sequential recommendation (MSR) incorporates textual and visual information to improve recommendation quality. However, recent studies and our empirical analysis show that visual features are often underutilized, thereby contributing far less than textual signals. We attribute this issue to two factors: insufficient visual representation learning (pretrained encoders fail to capture preference-relevant cues) and unbalanced visual-text optimization (textual features dominate the learning process). To address these issues, we propose Teach Multimodal Recommendation Model to See via Personalized Visual Extraction and Adaptive Learning (REVEAL), a plug-and-play framework that enhances visual representation learning and cross-modal optimization without modifying the original recommendation backbone. REVEAL consists of Feedback-Guided Visual Extraction (FVE), which refines prompt-guided visual extraction through task-level feedback, and Adaptive Visual Learning (AVL), which dynamically reweights visual learning to alleviate modality imbalance. Experiments on multiple real-world datasets and MSR backbones demonstrate that REVEAL consistently improves recommendation performance. Further analysis shows that these gains arise from more effective attention to preference-relevant visual regions and better visual utilization during training. The code is available at https://github.com/YutongLi2024/REVEAL.