Deep neural networks estimate repeated measurement regression functions
Generalized Deep Regression for Repeated Measurements
Machine Learning
Summary
The paper addresses the challenge of analyzing data where multiple measurements are taken on the same units, like people or machines, with responses that could be yes/no, counts, or continuous numbers. The authors study how deep neural networks can estimate the average relationship between input factors and outcomes while accounting for individual differences that are not directly observed. They derive mathematical guarantees on how accurate these neural network estimates can be based on the number of units and measurements. They also provide statistical tools to make reliable inference from the model and validate their methods through simulations and real-world data examples.
What this means in practice
- •For medical data analysts: Estimate average patient responses over multiple visits while accounting for individual differences in longitudinal health studies.
- •For manufacturing quality teams: Model repeated sensor readings from production units to improve understanding of process variations and trends.
Authors
Kexuan Li
Abstract
In this paper, we study the estimation of a marginal regression function from independent units with repeated binary, count, or continuous responses using ReLU deep neural networks. In the model, we assume that the dependence is generated by an unobserved random mean function within each unit. We then fit a neural network with a convex generalized regression loss. We show an oracle inequality by separating conditional measurement variation from between-unit variation. In addition, we prove that with $n$ units and $m$ measurements per unit, ReLU networks can attain an integrated mean squared error of order $n^{-1}+(nm)^{-2β/(2β+d)}$, up to logarithmic factors, over $β$-Hölder classes. We also derive a weighted oracle inequality for unequal cluster sizes and a rate for compositionally smooth functions. For pointwise ensemble inference, we give a projection central limit theorem and prove infinitesimal jackknife consistency under an explicit asymptotic linearity condition. Simulations and real data examples are provided to support our theoretical findings and practical implications.