Revisiting Shannon's Source Coding Theorem with Distributional Uncertainty under the Nonlinear Expectation Theory
2026-08-17 • Information Theory
Information Theory
AI summaryⓘ
The authors explore what happens when the usual assumption in information theory—that sources have fixed probability distributions—is relaxed, considering sources with uncertain distributions instead. Using a new mathematical framework called nonlinear expectation theory, they define a new kind of information entropy to measure how much information these uncertain sources hold. They also prove a source coding theorem showing limits on how efficiently these uncertain sources can be compressed. This extends classical information theory by better handling situations where probabilities themselves are not exactly known.
Information theoryProbability distributionNonlinear expectationUncertain-distribution sourcesInformation entropySource coding theoremStrong law of large numbersNonstationary processesSublinear expectationCoding rate
Authors
Wen-Xuan Lang, Shaoshi Yang, Jianhua Zhang, Zhiming Ma
Abstract
In classical information theory, a source is modeled by a single, precisely known probability distribution. However, in the increasingly complex communication networks full of unanticipated, nonstationary, and heterogeneous random events, the assumption of precise and well-defined probability distributions to describe random variables appears somewhat idealized. Therefore, it is important to characterize the uncertainty of distributions of source messages, subject to relaxing the assumption of deterministic probability models for analyzing information sources in information theory. Based on the nonlinear expectation theory, a novel axiomatical system that extends classical probability theory, this paper investigates the information sources whose distributions themselves are uncertain, and refers to them as uncertain-distribution sources. We generalize the fundamental concept information entropy to nonlinear information entropy, which describes the measurement of the amount of information contained in a uncertain-distribution source. By using the strong law of large numbers under sublinear expectation, we establish a nonlinear source coding theorem, which not only shows that the nonlinear information entropy is the upper bound for the infimum of achievable coding rate of uncertain-distribution sources under the maximum error probability criterion, but also determines a cluster point of the coding rate of uncertain-distribution sources under the minimum error probability criterion. Our findings reveal that the introduction of nonlinear expectation theory allows for a more comprehensive understanding of information sources.