Multi source prediction method improves accuracy across different data types
Multi-source conformal prediction: leveraging heterogeneity via localization
Machine Learning
Summary
Predictions often come from data collected in different places or conditions, which can vary a lot. This paper introduces a new method that combines information from multiple diverse sources to improve prediction accuracy, especially when new data doesn't match any single source well. The method adapts to where each data source is strongest and guarantees reliable predictions. Tests on simulated and real data show it works better than existing approaches.
What this means in practice
- •For machine learning engineers: Combine diverse datasets to produce more reliable predictive confidence intervals under varying test conditions.
- •For data integration specialists: Improve robustness of predictions when aggregating data from multiple heterogeneous sources with different feature characteristics.
Authors
Rohan Hore, Anirban Chatterjee, Sayantan Choudhury
Abstract
Many modern prediction tasks involve data from multiple heterogeneous sources, while the test distribution may differ substantially from any individual source. Although heterogeneity poses challenges, it also offers an opportunity: different sources may provide complementary information, with some regions of the feature space better represented in one source than another. We propose Multi-Source Randomly Localized Conformal Prediction (MS-RLCP), which builds on the local coverage properties of randomly localized conformal prediction (RLCP) (Hore and Barber, 2025) and extends it to multiple sources through data-adaptive source selection. Under the widely adopted assumption of a shared response distribution conditional on the features across sources and the test population, we establish finite-sample coverage bounds using an interpretable notion of envelope distribution that captures their aggregate feature-space representation. Our analysis allows the test feature distribution to be absolutely continuous with respect to the envelope, extending beyond mixtures of source distributions. Under additional regularity conditions, we also establish asymptotic test-conditional coverage. Simulations and real-world experiments demonstrate the effectiveness of MS-RLCP across varying levels of data heterogeneity.