User embeddings reveal clear behavioral patterns for better personalization
User Embeddings are Superpositions of Interpretable Behavioral Modes
Machine Learning
Summary
User embeddings are like digital fingerprints that capture how people behave online, but they are usually hard to understand because they mix many behaviors together. The authors show that these embeddings can be broken down into simpler parts that each represent a clear behavior mode, making them easier to interpret. They use special machine learning tools to find these behavior modes and even give them names using natural language. This helps to find and describe groups of users without needing extra text data. Their method works well on both fake and real user data and does not require changing the original embedding system.
What this means in practice
- •For digital marketing teams: Identify and label user groups based on clear behavior patterns to target personalized advertising more effectively.$Commercial implications: Enables selling targeted advertising solutions using interpretable user segments derived from embeddings for precise audience engagement.
- •For behavioral analytics engineers: Extract interpretable user behavior modes from embeddings to improve model transparency and explain user segmentation without retraining.
Authors
Jonathan Kozaczuk, Zheng Dong, Noyan Evirgen, Keivan Majidi, Eddie K. H. Ng, Chris Porter, Qi Zhao, Jia Geng, Changmao Li, Brandon R. Feng
Abstract
User embeddings are central components of modern recommendation, personalization, and behavioral modeling systems. Though these embeddings can compactly encode rich behavioral information, they are inherently dense and opaque, limiting interpretability. To address this gap, we argue that user embeddings can be understood as sparse superpositions of interpretable behavioral modes. This structure is directly predicted by generative models of user behavior for linear embeddings and extends approximately to deep-learning representations by the superposition hypothesis. Superimposed modes can naturally be recovered by sparse dictionary learning methods, such as sparse autoencoders (SAEs), and recovered modes can be automatically named and queried in natural language. We validate mode recovery against known ground truth on synthetic data, where SAEs recover modes better than classical sparse baselines, including on transformer-encoded embeddings. We then apply the method to classical and transformer-based user embeddings from two real-world datasets. We also show how the corresponding natural-language labels enable zero-shot audience retrieval and characterization over embeddings never trained on text. The framework is post-hoc and model-agnostic, requiring no modification to the upstream embedding system.