Image context improves recommendations when combined with other signals
Seeing the Context: Enhancing Recommender Systems with Image-Derived Contextual Signals
Information Retrieval
Summary
Recommender systems suggest things you might like based on clues like location or reviews. This paper shows that photos connected to a product or place can also give useful clues about the situation when someone chooses an item. The authors created a way to analyze these images for context and combined this with traditional clues to make better suggestions. Images alone were not as good, but adding them improved recommendations by giving different types of helpful information.
What this means in practice
- •For ecommerce platform teams: Enhance product recommendations by integrating image-derived situation clues alongside text and location data to improve customer choices.$Commercial implications: Enables platforms to sell recommendation features that use images as complementary context for better user engagement.
- •For travel app developers: Improve recommendations of attractions and services by combining image context with review data for richer user experience.
Tested on one dataset.
Authors
Tal Cordova, Tomer Geva, Moshe Unger
Abstract
Contextual information, capturing the circumstances of a user-item interaction, is central to recommender systems. Prior work draws context from location, time, or reviews, but not images; multimodal recommender systems mainly use images to enrich item or user representations, not identify situational context. We propose a new representation of context derived from images, spanning physical, social, and modal categories learned via a vision-language model. We introduce ICE-Fuse, a pipeline for evaluating this representation that fuses these categories and integrates them into a context-aware recommender system, using TripAdvisor data and Review-aware Graph Contrastive Learning as the recommendation algorithm. Image context does not outperform established signals standalone, but improves them combined, indicating complementary information. Semantic analysis shows image- and review-derived context capture distinct aspects of the interaction, positioning images as complementary context.