Regionfed improves personalized search in diverse retail stores
RegionFed: Federated Learning for Personalized Query Understanding in Heterogeneous Retail Environments
Machine LearningArtificial Intelligence
Summary
Online shopping search is tricky because people in different areas use different words and like different products, making it hard to create one model that works well everywhere. The authors found that usual privacy methods either miss local details or break when using modern AI models like transformers. They created RegionFed, a new way to train these models that checks how region-specific and general ideas conflict to decide how much to personalize for each area. This method works on various AI systems without changing their code and improves search results close to the best possible centralized method, while keeping data private.
federated learningpersonalizationtransformersquery understandingprivacygradient conflictT5 modelLayerNormdifferential privacymodel heterogeneity
Authors
Quoc H. Nguyen, Ali Lafzi, Abhijeet Phatak, Siddharth Pratap Singh, Rohit Upadhyay, Yogananda Domlur Seetharama, Chittaranjan Tripathy
Abstract
Retail search systems serve diverse geographic regions with distinct query patterns, vocabularies, and product preferences, creating significant data heterogeneity that challenges both privacy-preserving training and model personalization. Federated learning offers a natural solution for privacy, but standard FL methods produce global models that sacrifice regional performance, while existing personalized FL approaches operate at the parameter level and catastrophically collapse on modern transformers (below 10\% accuracy on T5) due to tied embeddings and LayerNorm interactions. We introduce RegionFed, an \textit{architecture-robust} federated learning framework that sidesteps this failure by operating entirely at the gradient level. RegionFed uses the $\ell_2$ conflict between regional and global gradients as a unified signal that (i) diagnoses heterogeneity, (ii) routes each region to the cheapest sufficient personalization strategy, and (iii) adaptively controls personalization strength. Because it treats models as differentiable black boxes, RegionFed deploys on T5-Small, T5-3B, RoBERTa, and CNN with zero code changes, providing large gains on transformers (where parameter-level methods collapse) and consistent improvements on CNNs. Across three public datasets (Amazon ESCI, Amazon Reviews, LEAF-FEMNIST) and four architectures, RegionFed-Meta achieves 92.27\%, closing the gap to the privacy-violating centralized upper bound (Centralized + Regional Weighting: 92.04\%, $Δ$=0.23pp, within 1$σ$) while providing $(ε{\approx}0.60)$-differential privacy and $\mathcal{O}(1/\sqrt{T})$ convergence.