Learning needs fewer samples when training on more domains
How Many Samples Are Enough for Learning Across Domains?
Machine Learning
Summary
It can be hard for computers to learn things that work well across different environments or groups called domains. The authors found a rule that shows how many examples computers need to learn well in each domain, depending on how many total domains are used during training. Specifically, they show that if you train on more domains, you need fewer examples from each one. This helps explain when datasets have enough data to teach computers and guides how to build better training collections. They also link learning inside one domain to how well the computer does in new domains.
What this means in practice
- •For machine learning engineers: Estimate how many examples to collect per domain to build datasets that improve model generalization across environments.
- •For data platform teams: Design training data collection strategies that optimize sample size per domain based on the number of domains to reduce costs.
A theory result. No direct application yet.
Authors
Hong Zheng
Abstract
Understanding the fundamental mechanisms of learning is essential for designing systems with strong generalization. Recent studies have shown that increasing the number of training domains, or enlarging the distribution shift among them, improves generalization when each domain contains sufficiently many data samples. However, the conditions under which the data samples can be considered sufficient remain unexplored. In this work, we fill this gap by establishing criteria for per-domain sample requirements based on the presented learning bounds. These criteria not only reveal an inverse linear scaling law between the number of training domains and the number of samples required per domain, but also explain the fundamental rationale behind the assumption of data sufficiency, thereby providing theoretical guidance for assessing the adequacy of existing datasets and constructing datasets. This differs from classical learning theory, as the number of samples required is highly dependent on the number of training domains. Additionally, we prove the close relationship between in-domain learning and out-of-domain generalization through the presented generalization bounds, and lastly discuss some key arguments.