Multi-task ranking model improves recommendation systems with shared and task-specific signals
Task-Blind No MORE: Multi-Task Information Flow in Unified Ranking Backbones
Information Retrieval
Summary
Recommendation systems suggest items like videos or products based on your interests, and they often need to solve several related tasks at once. The authors found that many current models handle all tasks only after the main processing, missing chances to improve by sharing task-specific information earlier. They created a model called MORE that keeps and updates both shared and task-specific information throughout all processing steps, leading to better suggestions. Tests on large datasets and a popular social app showed it increased user engagement and made recommendations faster.
Recommendation systemMulti-task learningFeature interactionSequence modelingBackbone modelAnchor tokensTask-aware information flowUser behavior sequencesA/B testingComputational efficiency
Authors
Yuchen Wang, Feng Niu, Qing Tan, Junting Lu, Baoxin Wu, Jun Gao
Abstract
Industrial ranking models for recommendation have scaled feature interaction and sequence modeling separately; recent architectures such as HyFormer and MixFormer unify both in a stackable backbone. Real-world recommender systems, however, nearly always require multi-task learning, yet existing unified architectures confine multi-task modeling to shallow post-backbone towers, leaving the backbone without task-aware information flow. We propose MORE (Multi-task cO-evolving Ranking modEl), which embeds multi-task information flow inside the backbone, enabling task-specific signals to co-evolve with sequence and feature representations at every layer rather than in a post-hoc fusion. It introduces Anchor Tokens that persist across backbone layers: Shared Anchors encode cross-task commonalities, while Private Anchors capture task-specific priors. In each block, Anchor Tokens (1) read task-conditioned signals from behavior sequences, (2) mix with non-sequential features under a task-boundary mask, and (3) refine per-task representations through independent branches; as blocks stack, each task obtains a differentiated representation refined through all backbone layers. Experiments on large-scale industrial datasets show that MORE consistently outperforms baselines across all tasks under comparable parameter and FLOPs budgets, and scales well with model size. Online A/B tests on Momo, a leading Chinese social discovery platform with tens of millions of monthly active users, yield 3% improvement in usage duration, 3.6% in interaction rate, and 2% in deep-chat rate. MORE is deployed in production with request-level shared computation reducing scoring latency by about 30%.