Learning from label proportions improved by matching statistical moments
Rethinking Learning from Label Proportions via Moment Matching
Machine Learning
Summary
Sometimes, we want computers to learn from groups of data labeled only by their overall mix of types, not by exact labels for each item. Previous methods assumed the groups were randomly made, but this is often not true. The authors show that under a more realistic setup, the label counts follow a certain statistical pattern called the Poisson multinomial distribution. They propose a new method that trains a classifier by matching multiple statistical aspects (moments) of these groups, leading to better predictions for individual items within groups.
What this means in practice
- •For data scientists: Train classifiers more accurately when only group-level label proportions are available and individual labels are uncertain or missing.
- •For marketing analysts: Improve customer segmentation models when only aggregate category data by groups is collected, not individual customer labels.
Authors
Tianhao Ma, Wei Wang, Yivan Zhang, Dong-Dong Wu, Takashi Ishida, Gang Niu, Masashi Sugiyama
Abstract
Learning from label proportions (LLP) is a weakly supervised setting where training data are grouped into bags with only bag-level label proportions observed, and the goal is to learn a classifier that predicts labels for individual instances. Recently, a large body of methods has emerged; however, most of them rely on the assumption that instance-label pairs are sampled i.i.d. and randomly assembled into bags, an assumption often violated in real-world scenarios. Our experimental results indicate that these methods perform suboptimally under non-random settings. In this work, we adopt a more realistic assumption: bags are sampled i.i.d., and instance labels are conditionally independent given the instances. Under this setting, we show that the label counts follow a Poisson multinomial distribution. Motivated by this observation, we propose LLP via moment matching (LLP-MM), a simple yet effective approach which leverages multi-order factorial moments as training objectives, encouraging the classifier predictions to match these moments and thereby more fully exploit the underlying statistical structure of the data. Extensive experiments on benchmark datasets under various bag construction strategies demonstrate the effectiveness of our approach while maintaining high computational efficiency. Our code is available at https://github.com/TianhaoMa5/LLP-MM.