Unified model predicts mixed types of related outcomes accurately
Generalized Engression Models
Machine Learning
Summary
Many models predict one type of outcome, like numbers or categories, but the authors handle multiple types at once, such as numbers, yes/no answers, and rankings, that depend on each other. They created a new method called generalized engression models, which uses deep learning to estimate the full range of possible outcomes together. Their method works well even when outcomes are very different and matches or beats specialized models on complex data about species and health. This helps make better predictions when multiple types of outcomes are linked.
What this means in practice
- •For ecology modelers: Predict joint species distributions with mixed data types more accurately using a unified model instead of separate type-specific ones.
- •For health data scientists: Analyze complex multidimensional health outcomes involving continuous, categorical, and binary data jointly to improve distributional insights.
Authors
Xinwei Shen, Zijian Guo, Francis Bach
Abstract
We consider estimating the conditional distribution of a multivariate outcome given covariates when its coordinates may be continuous, binary, categorical, ordinal or rankings, and are conditionally dependent on one another. Different statistical methods have been developed for each outcome type, and most of them target a summary of the conditional distribution, such as the mean of each coordinate, rather than the joint distribution of the outcome vector. We develop generalized engression models, a unified nonparametric distributional regression framework for outcomes of any type. The proposed method builds upon engression, a scoring-rule-based deep generative model, and introduces a data-type-specific link function and a stochastic perturbation that smooths the loss, enabling gradient-based training even with discontinuous links. We establish universal representation results for continuous, discrete and mixed outcomes. In simulations and in two applications, 242 species in a community ecology benchmark and a 17-dimensional mixed-type health outcome, the method matches type-specific models on marginal scores, improves on them on the joint distribution, and matches or exceeds purpose-built state-of-the-art joint species distribution models. Software is available in Python.