Summary
Large language models can recommend items by generating lists from what they learn about users and items. This study shows that how you prepare the training data before fine-tuning these models changes how well they recommend and how fair the recommendations are across different groups. The authors found that including original item titles helps recommendations work better and balancing data for some demographics can reduce gaps between groups without hurting performance. However, balancing for gender didn't improve fairness and actually reduced usefulness. The study suggests that data preparation affects both accuracy and fairness and should be considered carefully together.
What this means in practice
- •For ai product teams: Ensure recommendation models avoid invalid outputs and improve fairness by carefully choosing data preprocessing steps during fine-tuning.
- •For marketing analytics teams: Improve movie or product recommendation accuracy by including original item titles and demographic balancing in training data.
Tested on one dataset.
Abstract
Large language models (LLMs) have introduced a generative approach to recommendation, producing item lists from textual representations of user histories and item information. However, this generative setting raises challenges concerning output validity, prompt consistency, and sensitivity to demographic descriptors. In this work, we investigate how preprocessing choices in fine-tuning data affect recommendation utility and fairness-related behavior in a generative LLM-based recommender system. We first establish a controlled recommendation pipeline that combines supervised fine-tuning, catalog-constrained decoding, and prompt augmentation. We then study three preprocessing choices: item-title representation, demographic balancing, and auxiliary item-knowledge enrichment. Experiments on MovieLens-1M show that the catalog-constrained pipeline eliminates out-of-catalog items and ensures complete ten-item outputs, while prompt augmentation improves prompt consistency and utility. Original MovieLens titles increase HR@10 from 10.67% to 12.47% and Coverage@10 from 41.36% to 47.88%. Fairness-related effects are attribute-dependent: age balancing and knowledge enrichment reduce selected group-level utility gaps while largely preserving utility, whereas gender balancing reduces utility without improving the evaluated gender-related indicators. Overall, preprocessing choices in fine-tuning data should be evaluated jointly in terms of utility, output behavior, and demographic sensitivity.