Dual branch modeling improves multimodal multi scenario recommendations
DuTri: Dual-Branch Modeling with Triadic Factorization for Multimodal Multi-Scenario Recommendation
Information Retrieval
Summary
Recommendation systems often need to suggest items based on a mix of different scenarios and types of information like text and images. The authors found that simply combining these inputs in traditional ways doesn’t work so well. They designed a new method called DuTri that uses two separate parts: one to understand user preferences across different scenarios, and another to handle and combine the text and image data effectively. Their approach helps the system make better recommendations by aligning these different kinds of information. They tested DuTri on real-world data and saw it perform better than existing methods.
What this means in practice
- •For ecommerce platform engineers: Enhance product recommendation algorithms by jointly modeling user preferences and multimodal content across different shopping scenarios.$Commercial implications: This paper enables better personalized recommendations on ecommerce sites by integrating multimodal data effectively, improving user engagement and sales.
- •For online content recommendation teams: Develop more accurate cross-scenario content suggestions by separating collaborative signals from multimodal semantics for adaptive fusion.
Authors
Jintao Huang, Chaohua Yang, Kailiang Hao, Dugang Liu, Xing Tang, Zhong Ming
Abstract
As recommendation scenarios become increasingly diverse and multimodal information grows more prevalent, effectively modeling cross-scenario user preferences and multimodal semantics has become a critical challenge in recommender systems. When incorporating multimodal information, existing multi-scenario recommendation (MSR) methods typically treat textual and visual representations as generic input features and directly feed them into their original parameter-sharing architectures. However, this extension may not fully accommodate the different modeling requirements of collaborative and multimodal information and may not be the best modeling paradigm. Our controlled experiments show that directly adopting these architectures is not the optimal modeling approach. To this end, we propose Dual-Branch Modeling with Triadic Factorization (DuTri), a framework for multimodal multi-scenario recommendation comprising a collaborative branch and a multimodal branch. Specifically, the collaborative branch learns structured collaborative preference representations through CANDECOMP/PARAFAC (CP) decomposition of user-scenario-item interactions and residual quantization, thereby capturing cross-scenario commonalities and scenario-specific differences. The multimodal branch leverages the learned user and scenario preferences to guide adaptive modality fusion; additionally, item-level contrastive learning aligns collaborative item preference representations with multimodal representations to facilitate effective collaboration between the two branches. Extensive experiments on two real-world datasets, AntM$^2$C and Amazon, demonstrate that DuTri outperforms existing multimodal and multi-scenario recommendation baselines in overall recommendation performance, validating the effectiveness of the proposed framework.