Unified distributional regression with ReLU neural networks for complex data
Distributional Regression for Arbitrary Response Variables Using ReLU Neural Networks
Machine Learning
Summary
Predicting how an outcome spreads or varies can be tricky when the outcome is complicated or doesn’t fit usual categories. This paper offers a new way to predict full probability distributions instead of just single numbers, even when the outcomes live in unusual spaces. The authors use neural networks with ReLU activation functions to estimate these distributions by looking at events as sets rather than simple thresholds, improving flexibility and accuracy. Their method comes with mathematical guarantees about accuracy and performs well on both simulated and real data.
What this means in practice
- •For data science teams: Model complex outcome distributions in non-standard spaces with neural networks for better probabilistic predictions.
- •For healthcare analytics teams: Estimate distributions of varied medical outcomes beyond standard numeric measures, improving risk assessment.
Authors
Junpeng Ren, Carlos Misael Madrid Padilla, Oscar Hernan Madrid Padilla
Abstract
This paper develops a unified framework for distributional regression with arbitrary response variables. Rather than relying on scalar thresholds or Euclidean structure, we model conditional event probabilities indexed by measurable sets, which extends distributional regression to general response spaces while retaining a direct connection to classical event-based formulations. Within this framework, we study two complementary estimators: a preliminary estimator based on separate regression of event indicators, and a constrained estimator that enforces inclusion-consistent predictions through a directed acyclic graph (DAG) over a finite set family. We establish general error bounds for both estimators and further specialize the results to dense ReLU networks, yielding explicit convergence rates overcoming the curse of dimensionality. Simulation studies and real-data experiments further demonstrate the practical effectiveness of the proposed framework.