Multitask regression models adapt coefficient sharing by predictors
Multitask Regression with Pairwise Fusion
Machine Learning
Summary
When predicting many things at once, some features may relate to most predictions similarly, while a few relate differently for certain cases. The authors study a way to model this by grouping coefficients that share values across tasks but allow exceptions that can vary by feature. They penalize differences between coefficients for better estimation and identify when many tasks can share the same model. Their approach works well on simulated and real household energy data, showing how sharing patterns can flexibly change.
What this means in practice
- •For data scientists: Improve multitask predictive models by tailoring coefficient sharing per feature, capturing both common and exceptional relationships across tasks.
- •For energy analysts: Enhance forecasting of household energy consumption by recognizing which features share effects across households and which differ individually.
Authors
Xiaodong Li, Zhentao Li
Abstract
We study multitask regression when coefficient sharing can differ by predictor. For a given predictor, many tasks may have the same coefficient while a few differ, and the exceptional tasks need not be the same for another predictor. We describe this structure by two quantities: the number of active predictors and the total number of task coefficients that differ from the most common value for their predictor. We estimate the coefficient matrix by penalizing all pairwise coefficient differences across tasks, with an additional group penalty when predictor selection is needed. The resulting upper and lower bounds have the same dependence on these two quantities. We also consider the stronger setting in which a large set of tasks shares one entire coefficient vector. Under explicit sample-size conditions, the same pairwise estimator pools those tasks exactly, while allowing the remaining tasks to differ. Simulations and household energy data illustrate the transition between broad sharing and task-specific coefficients.