Summary
Recommender systems often guess what people like based on past behavior, but it can be hard to see or change why they make certain suggestions. The authors studied a system that creates user profiles written in plain language from reviews people write, letting users directly tweak what the system thinks they like. They confirmed this approach works well and makes recommendations clearer, but found changing these profiles affects ratings uniformly rather than changing the ranking of different genres. The study also showed that how the system is trained on ratings influences this behavior more than the natural language profiles themselves.
What this means in practice
- •For e-commerce platform engineers: Adapt recommendation interfaces to let users directly edit natural-language profiles to better understand and customize suggested products.
- •For travel booking service developers: Use natural language profiles from user reviews to provide transparent, interactive hotel recommendations addressing cold-start users.
Abstract
In this reproducibility study, we investigate the transparency and scrutability of recommender systems enhanced by incorporating generated natural-language user profiles that represent user preferences. The original paper explores the synthesis of user profiles from raw user-generated review text across domains such as movies and accommodations (Amazon Movies & TV, TripAdvisor). Crucially, these natural-language user profiles enable direct user interaction and intervention, allowing users to customize recommendations by correcting misattributed preferences or addressing cold-start settings. We successfully reproduce the core findings of the original study. Additionally, we extend the evaluation by conducting systematic context ablation experiments, multi-seed stability across five distinct random seeds to establish statistical reliability, and a mechanistic interpretability analysis using the nnsight framework to probe internal model representations under counterfactual profile perturbations. Our findings verify the original paper's claim that User Profile Recommendation (UPR) achieves competitive performance under its test-set reranking protocol and makes recommendations more transparent. Perturbing the natural-language profiles does change predictions, but it shifts predicted ratings uniformly across genres with no detectable genre-selective effect, leaving rankings unchanged even under direct activation steering. We trace this back to the rating-regression objective rather than the profile interface, with ranking-objective models clearly exceeding in this task.