Multimodal graph method improves item recommendations efficiently
Multimodal Graph Retrieval-Augmented Sequential Recommendation via Collaborative Filtering Paths
Machine Learning
Summary
Making good recommendations often means understanding what a user likes and what similar users enjoy. The paper presents a new way to improve recommendations by combining information about one user with signals from other users nearby in a network, especially when items have images, text, or other data types. Their method finds helpful connections in a user-item network and feeds those to a large language model just once per item, making the process faster and better. Tests show this method predicts what users will like more accurately than past approaches.
What this means in practice
- •For ecommerce platform engineers: Improve product recommendation ranking by integrating collaborative signals and multimodal data into language model prompts with fewer network calls.
- •For media streaming service teams: Enhance sequential suggestions for users by retrieving relevant interaction paths weighted by multimodal similarity without repeated model inference.
Authors
Jason Marcell Setiadi, Xin Cao, Lina Yao
Abstract
Multimodal Large Language Models (MLLMs) have demonstrated strong potential for sequential recommendation through their ability to reason over complex multimodal data. However, existing approaches either rely solely on the target user's own interaction history, neglecting collaborative signals from neighboring users, or incur substantial computational overhead through repeated MLLM inference over long interaction histories. To address these challenges, we propose MGRASRec, a multimodal graph retrieval-augmented framework for sequential recommendation. MGRASRec injects collaborative filtering signals conditioned on the candidate item directly into the MLLM prompt by retrieving structured paths from a user-item interaction graph, extended via multimodal similarity to increase coverage beyond exact co-interaction overlap. This retrieval also surfaces the history items most relevant to the candidate at no additional cost, removing the need for recurrent summarization and keeping inference to a single forward pass per candidate. All components are unified into an augmented prompt for parameter-efficient fine-tuning of an MLLM. Extensive evaluations across three publicly available datasets validate the effectiveness of MGRASRec, achieving the best performance on all metrics with particularly strong gains in ranking quality.