Shared representations face limits in multitasking capacity
The Bell Boundary of Shared Representations
Machine LearningComputation and Language
Summary
Multi-task learning models try to solve several problems at once using a shared way of understanding information. The authors show there is a clear boundary where sharing this understanding stops working well because tasks interfere with each other. They use a concept called Bell's theorem to measure when this happens and introduce a model to study how the capacity of the shared system affects task compatibility. Their experiments show that when the system's capacity is limited, tasks can clash, but with enough capacity, this clash disappears.
What this means in practice
- •For machine learning engineers: Identify capacity limits in multi-task models to optimize shared representation design and task allocation.
- •For ai system architects: Use the Bell Boundary diagnostic to measure and manage task incompatibility in multi-task AI deployments.
Authors
Yang Wu, Hanyu Zhao, Yazhou Zhang, Yuexian Hou
Abstract
When does a shared representation become insufficient for jointly supporting multiple tasks? This question lies at the heart of multi-task learning models. Although task interference in shared representations has been widely observed, its onset, measurable form, and capacity boundary remain poorly understood. In this paper, we revisit this problem through the lens of Bell's theorem. We show that if four binary task outcomes can be jointly represented by a common shared representation, their correlations must satisfy the classical CHSH bound $|S|\leq2$. We introduce NCnet, a controlled multi-task model for studying how representational capacity affects task compatibility. Experiments reveal a clear capacity-dependent transition. Once nontrivial task fitting emerges, constrained capacity can yield $|S|>2$. As capacity further increases, these exceedances progressively disappear and $S$ returns to the bounded region. This systematic pattern provides evidence that limited representational capacity is an important source of the observed incompatibility. We observe similar transitions across CNNs, PLMs, and LLMs, suggesting that the phenomenon extends beyond the controlled NCnet setting. We refer to this capacity-dependent transition as the Bell Boundary, providing a measurable diagnostic for representation bottlenecks, task incompatibility, and capacity allocation in multi-task models.