Papers for

recommendation engineers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

User activity can create false item similarity in recommender systems

Do Similarity Measures Measure Similarity? How User Activity Creates Phantom Correlation in Recommender Systems

Abstract: Item-item similarity is one of the oldest and most foundational signals in recommender systems: two items are assumed to be similar when the same users consume them. Despite the sparsity and scale of interaction data, the implicit assumption across the field is that item co-occurrence reflects genuine shared preference, capturing what users like together. In this paper, we demonstrate that a substantial fraction of observed item-item similarity is an artifact of variance in user activity rather than shared taste. Even if every user consumed items completely independently, differing only in their total consumption volume, standard similarity measures still detect strong associations between items. We term this phenomenon phantom correlation: spurious item association that emerges strictly from heterogeneous user activity in the complete absence of preference structure. Starting from minimal formal assumptions, we show that independent consumption under varying activity is described by a Rasch model, and we derive how the resulting association depends on item popularity for each similarity measure. By comparing our theoretical baseline against real-world benchmark datasets, we reveal the extent to which popular similarity measures reflect phantom correlation, which bears on how item co-occurrence is interpreted and used in collaborative filtering.

Mon 26 OctInformation RetrievalArtificial Intelligence
The gist
Recommender systems often assume that when many people use two items, those items are similar because people like them together. The authors show that a lot of these detected similarities are actually fake and come from some users just being more active than others, not from real shared preferences. They call this problem phantom correlation and explain it with a statistical model. This means some item similarities that recommendation algorithms rely on might not reflect what users truly like.
Open → 2610.35765v1

New training method improves click prediction accuracy with one network

FLMNet: Self-Ensemble Distillation from Masks to Seeds for Single-Tower CTR Prediction

Abstract: Click-through rate (CTR) prediction underpins recommender systems and online advertising, where high-dimensional categorical features interact in combinatorially many ways. Several recent CTR models, such as FinalNet and QNN, report higher accuracy when a network also learns from its own averaged predictions, for example by running the same network under different dropout patterns and training each run toward the average. Why this self-distillation helps, and whether its random sampling is needed, remains unclear. This work shows that as the number of dropout patterns grows without bound, the objective reduces exactly to the label loss of the logit-averaged prediction plus a penalty on how much the dropout copies of the network disagree. When dropout does not affect how a network gates its features, this penalty can be approximated in closed form, and FLMNet is designed on this principle and trains without sampling. Networks trained from different seeds disagree in the same form, and distilling several of them into one network recovers part of their ensemble gain. On five benchmarks against seventeen baselines, the resulting models take the top two places in accuracy with a single network at inference, and the gains of the penalty and of distillation, where measurable, largely add up.

Mon 19 OctInformation RetrievalMachine Learning
The gist
Predicting whether people will click on ads or recommendations is hard because there are many ways features can interact. The authors studied a technique where a network learns from multiple versions of itself, each slightly different due to dropout, and found that this training approach implicitly encourages agreement between these versions. They designed a new model called FLMNet that uses this insight to train without needing to sample those versions randomly. Tests on five different tasks showed FLMNet often beats stronger models while only using one network when making predictions.
Open → 2610.26143v1

Language embeddings can leak item and user identities in recommenders

Encoded but Exposed: Language Embeddings can Reveal More Than They Should in Recommender Systems

Abstract: Language embeddings are increasingly used in recommenders to represent item content, but their privacy risks remain poorly understood. We study whether item embeddings can reveal item identities when an adversary has access to auxiliary item text from external sources. We show that a simple cosine-similarity attack can recover item identities with high mean reciprocal rank across language models and candidate-set settings on two real-world datasets. Leakage is strongest when the adversary's partial information overlaps with the victim representation, but it also persists through semantic alignment between independently sourced descriptions. We further show that item leakage can propagate to user re-identification from sparse ratings. Finally, we evaluate obfuscated descriptions as a mitigation and find a clear leakage-utility tradeoff.

Thu 15 OctCryptography and SecurityArtificial Intelligence
The gist
Language embeddings are used by recommendation systems to understand items based on their text descriptions. This paper shows that these embeddings can unintentionally reveal the exact identity of items if someone compares them with known text information from other sources. Even when the descriptions are different but related in meaning, the risk remains. Furthermore, this information leak can help identify users based on their item ratings, posing privacy concerns. The authors also find that hiding some details in descriptions reduces leakage but affects recommendation quality.
Open → 2610.21713v1

Recommendation system improves cold-start item suggestions using dual-branch graph learning

Bridging the Topological Divide: Dual-Branch Graph Distillation with Semantic Priors for Robust Cold-Start Recommendation

Abstract: Graph Collaborative Filtering models have achieved remarkable success in recommendation systems. However, they inherently suffer from a catastrophic representation collapse in strict cold-start scenarios due to the topological isolation of unobserved items. While integrating multi-modal semantic features offers a potential remedy, directly mapping isolated semantic attributes into a globally topology-driven collaborative space frequently precipitates representation degradation and optimization conflicts. To bridge this fundamental structural divide, we propose DualGCN, an innovative Dual-Branch Graph Knowledge Distillation Framework. Operating under a decoupled two-stage training paradigm, DualGCN first constructs a robust collaborative teacher network where item representations are anchored by a topology-smoothed semantic prior, effectively reshaping the latent manifold. Subsequently, a semantic student branch employs a residual mapping mechanism and a magnitude-agnostic cosine distillation objective to achieve high-fidelity cross-modal alignment without disrupting established collaborative signals. Furthermore, we introduce a dynamic norm reconstruction mechanism to rigorously calibrate the physical scales of novel and warm items during global retrieval. Extensive experiments across three real-world benchmark datasets demonstrate that DualGCN significantly outperforms state-of-the-art baselines in strict cold-start scenarios while comprehensively preserving recommendation accuracy for highly active items.

Wed 14 OctInformation RetrievalMachine Learning
The gist
Recommendation systems struggle to suggest new items that have no user interaction history, resulting in poor performance for these 'cold-start' items. The authors propose DualGCN, a two-part method that first creates strong item representations using network information and semantic features, then carefully aligns semantic information without disturbing the network-based knowledge. This approach helps the system understand new items better without losing accuracy on popular items. Experiments show their method works well on real datasets.
Open → 2610.19578v1

Embedding subspaces enable flexible multi-goal recommendations at scale

Embedding Subspace Partitioning for Dynamic Multi-Objective Retrieval

Abstract: Modern industrial recommender systems must optimize across competing objectives, balancing semantic relevance with business metrics such as engagement and revenue. While bi-encoders dominate large-scale retrieval due to their efficiency, they collapse these heterogeneous signals into a single static embedding space. This design creates a fundamental limitation: once trained, the retriever cannot adapt to shifting objective priorities at serving time without retraining. Moreover, joint optimization with multi-objective losses often induces interference between objectives, leading to suboptimal trade-offs. We propose Embedding Subspace Partitioning (ESP), a retrieval framework that decomposes the embedding into task-aware subspaces and replaces the single dot product with a weighted sum of per-subspace similarities, whose weights are tunable at serving time. For Transformer bi-encoders, ESP uses the model's native end-of-sequence token as a segment delimiter, with segment-aware attention masking and position encoding resets to guarantee subspace isolation in a single forward pass. Serving is performed via GPU-accelerated exhaustive kNN over one concatenated index, eliminating the need for per-objective Approximate Nearest Neighbor (ANN) infrastructure required by multi-head approaches. We evaluate ESP on an open-source benchmark built from MS MARCO. A single ESP model traces a broad Pareto frontier, consistently outperforming strong multi-task baselines across diverse operating points. In LinkedIn's job matching platform (70M+ weekly users), ESP enabled dynamic retrieval reconfiguration and delivered significant key business metric lifts.

Thu 24 SeptInformation Retrieval
The gist
Recommender systems often need to balance different goals like showing relevant items and increasing business value, but traditional methods mix all goals into one fixed understanding. The authors propose splitting the understanding into separate parts, each focused on a specific goal, so the system can adjust priorities on the fly without retraining. Their method works efficiently on large-scale systems by doing one search over combined data rather than multiple searches. Tests show this approach better balances goals and improved user engagement in a large job matching platform.
Open → 2609.30601v1

Multi task learning improves recommendation accuracy and satisfaction

Learned Cross-Task Relationships in Multi-Task Models

Abstract: We propose a framework that learns cross-task relationships in multi-task models by approximating the joint distribution of task labels through targeted pairwise relationships. This approach improves performance via transfer learning and enhances information extraction without the intractable complexity of modeling the full joint space. Although our framework applies to any multi-task system, we demonstrate its efficacy within YouTube's production recommendation systems. Experiments across the Notifications, Homepage, and Watch Next surfaces show improvements in both accuracy and user satisfaction metrics. Finally, we propose a workflow template to facilitate broader future implementation.

Wed 23 SeptArtificial Intelligence
The gist
Many online services try to do several related tasks at once, like recommending videos and sending notifications. The authors introduce a way to help computers understand how these tasks relate to each other by looking at pairs of tasks rather than everything at once. This approach makes the system better at learning and sharing useful information, improving recommendation performance. They tested their method on YouTube’s recommendations and found it made users happier with the suggestions they received.
Open → 2609.28776v1

Dataset recommender system helps choose evaluation data for recommender algorithms

FINALLY: A Dataset Recommender System for Recommender-Systems Research

Abstract: Dataset selection shapes the empirical conditions under which recommender-system algorithms are evaluated, yet existing tools provide limited support for constructing complete dataset sets that jointly satisfy experimental constraints and set-level selection objectives. To address this problem, I developed FINALLY, a web-based dataset recommender for constructing configurable dataset sets for offline recommender-systems evaluations. FINALLY combines required datasets, candidate-pool restrictions, metadata filters, configurable target-set sizes, Random selection, and diverse and non-diverse strategies based on adapted Effective Covariance and Convex Hull objectives. I evaluated FINALLY through 420 recommendation runs across ten systematically varied configurations. All evaluated dataset sets satisfied the applicable target-size, duplicate-avoidance, snapshot-membership, required-dataset, and metadata-filter requirements. All 40 deterministic strategy--configuration combinations were reproducible. Both the Effective-Covariance-based and Convex-Hull-based strategies produced the expected diverse-versus-non-diverse score ordering in all ten configurations. Under their corresponding objectives, the diverse strategies produced scores above all 30 configuration-specific Random results, whereas the non-diverse strategies produced scores below all 30 Random results. These results establish technical consistency for the evaluated FINALLY workflow and show that the implemented strategies follow their intended optimization directions within the investigated configuration space. They do not establish the scientific suitability, global optimality, or practical superiority of the generated selections.

Tue 8 SeptInformation Retrieval
The gist
Choosing the right datasets to test recommendation algorithms is important but challenging. The authors created FINALLY, a web tool that helps pick groups of datasets meeting specific rules and goals. They tested it thoroughly and found the tool consistently selects datasets as intended, including diverse and non-diverse groups. However, they note this does not prove which selections are best for all uses. The tool helps system builders set up fair and varied testing scenarios for recommendation algorithms.
Open → 2609.08941v1