Consideration circuits improve multi-stage choice modeling with deeper networks
Consideration Circuits: Depth Separation and Universality Beyond a Single Softmax
Machine Learning
Summary
Most models that predict how people choose between items use a single step to score each option, but this can limit accuracy. The authors introduce consideration circuits, which build multi-step models that combine choices in stages using networks called multinomial logit units. Their results show that adding more depth to these circuits greatly improves the model’s ability to fit complex choice behaviors and reduces error. Experiments also show these circuits perform better than existing models on various real-world choice datasets.
What this means in practice
- •For online travel platforms: Build better recommendation systems by modeling user choice with multi-stage consideration circuits to improve prediction accuracy for hotel and flight selections.$Commercial implications: Enables travel platforms to offer more relevant choices by better capturing complex user preferences, potentially increasing bookings and revenue.
- •For e-commerce product teams: Enhance product ranking algorithms by using consideration circuits to capture multi-step customer decision processes, improving recommendation relevance on fixed product pools.
Authors
Junjie Xiao, Huiwen Jia
Abstract
Most feature-based choice models, classical and deep, score items and apply a single softmax. We introduce consideration circuits (CC), feature-based models of multi-stage choice defined by directed acyclic graphs of multinomial logit (MNL) units. Source units assign probabilities to menu items, and internal units combine predecessor distributions using MNL weights computed from their probability-weighted feature summaries. On a three-item compromise task with fixed non-collinear features, menu-independent random-utility models (RUM), including a single MNL unit, suffer an error bounded away from zero. For CC, in contrast, we establish a sharp depth--norm separation: increasing depth from $2$ to $3$ reduces the optimal maximum taste-vector norm for error $ε$ from $Θ(\log(1/ε)/ε)$ to $Θ(\log(1/ε))$. The depth-$2$ lower bound holds for arbitrary width and menu-independent routing biases, while a five-node depth-$3$ circuit with zero routing biases attains the logarithmic rate. More generally, we characterize two geometric conditions that are necessary and sufficient for approximating arbitrary deterministic choice tables on finite menu families. Under these conditions, depth $3$ suffices, while depth $4$ achieves optimal logarithmic norm scaling whenever the family contains a non-singleton menu. In experiments, standalone tree circuits with fewer than $600$ parameters attain the lowest mean test negative log-likelihood (NLL) among the evaluated models on four fixed-pool benchmarks and the Expedia temporal split. As output heads, CC generalize the linear MNL readout and lower mean test NLL for every tested encoder on Expedia and Trivago.