Joint modeling improves prediction of ordinal preferences with covariates
Covariate-dependent Joint Modeling of Multivariate Ordinal Preferences and Its Connections with Comparison Models
Machine Learning
Summary
People often rate things like movies or chatbot responses using scales rather than simple thumbs up or down. Current models usually treat each rating separately or reduce ratings to simple comparisons, losing information. The authors created a new method that looks at all ratings together and includes extra information about the rater or item. This approach keeps more detail from the data and can model preferences better, even when some calculations are tricky.
What this means in practice
- •For recommender system engineers: Build recommendation tools that jointly analyze multiple item ratings and user traits for more accurate predictions.
- •For machine learning practitioners: Improve alignment of large language models by jointly modeling multiple human feedback scores and prompt-response features.
Authors
Yujie Chen, Antik Chakraborty, Anindya Bhadra
Abstract
Multivariate ordinal data along with covariates are commonly collected in problems ranging from alignment of language models with human preferences, as well as in recommender systems. For example, data sets such as MovieLens contain several movies rated on a scale 1--5 by human users, along with their demographic information such as age or gender. Similarly, data sets such as HelpSteer collect human feedback on several attributes such as "helpfulness" or "verbosity" of LLM response on an ordinal scale, with covariates depending on the LLM prompt--response pairs. Unfortunately, the standard approaches for modeling these data (a) look at the attributes individually rather than jointly, and (b) often convert the data into pairwise or list-wise win--loss comparisons for fitting models such as Bradley--Terry and Plackett--Luce. Both of these lead to a coarsening of what is actually observed, which we address via a joint covariate-dependent consecutive ratio Markov random field model. We also show pairwise or listwise comparison models are obtained under restrictions of our joint model, and that joint modeling improves comparisons. We also develop a maximum likelihood inference procedure even in the presence of an intractable normalizer.