Numerical embeddings improve click prediction with stable value contexts
ScalarLens: Numerical Embeddings with Stable Coordinates and Contextual Responses for CTR Prediction
Information Retrieval
Summary
Predicting whether someone will click on an ad is tricky because numbers can mean different things depending on the situation. The authors found that treating each number as having only one meaning breaks this prediction. They created ScalarLens, a way to keep a number’s stable identity but change how the prediction reacts to it depending on context. This method worked better than many other approaches across different tests, helping computers better understand when a number means more or less for predicting clicks.
What this means in practice
- •For advertising platform engineers: Improve online ad click prediction by embedding numerical features with context-sensitive responses that ensure stable value interpretation.
- •For financial risk modeling teams: Enhance models by embedding numerical inputs that keep stable meaning while adapting to changing data contexts, improving prediction accuracy under distribution shifts.
Authors
Heng Yao, Tianying Liu, Yulou Shu, Yong He, Chuan Yuan, Kaibin Qiu, Guowei Chen, Jiayu Zhao, Siyun Hou
Abstract
Numerical embeddings for click-through rate (CTR) prediction are built on a convenient but restrictive premise: a scalar has one representation. This premise conflates where a value lies with what it means for the current sample. On the Criteo validation split, the same numerical interval carries residual click evidence with opposite signs across categorical and numerical contexts, even after additive main effects are removed. Production pipelines compound this mismatch because externally normalized features require transformations and statistics to remain synchronized between training and serving. We introduce ScalarLens, a numerical embedding that preserves what a value is while adapting how it should be interpreted. A monotone local mesh constructs a stable coordinate from the focal scalar alone; bounded low-rank dynamics then produce a contextual response without moving that coordinate or replacing categorical tokens and the CTR backbone. In a 1,539-run primary evaluation covering 19 representations, three datasets, nine backbones, and three seeds, ScalarLens ranks first in 25 of 27 settings on original numerical scales and second in the remaining two. Matched ablations show that scale correction, additional local capacity, and generic conditioning do not reproduce the gain. A controlled study further recovers categorical, numerical, and mixed response mechanisms under context shift while the focal coordinate remains exactly invariant. A complete rerun under shared standardization retains significant advantages over DEER, DAES, and NaryDis, showing that the result is not explained by tolerance to raw scales alone. ScalarLens therefore recasts numerical embedding as a measurement problem: coordinates belong to values, while predictive responses belong to values in context.