Cross-Domain Lossy Compression via Constrained Minimum Entropy Coupling
2026-05-11 • Information Theory
Information TheoryMachine Learning
AI summaryⓘ
The authors study how to compress data from one domain and reconstruct it in another domain while keeping important information for classification tasks. Instead of focusing on pixel-level differences, they use a method based on information theory called minimum entropy coupling to keep the source and output strongly related. They develop mathematical solutions for simple cases and propose a neural network approach for more complex data. Their experiments on image tasks show that allowing more compression bits improves both classification accuracy and the quality of the reconstructed images.
cross-domain compressionminimum entropy couplingrate constraintclassification constraintlogarithmic lossBernoulli sourcesinformation theoryneural restorationentropy modelingdistribution matching
Authors
Nam Nguyen, Hassan Tavakoli, An Vuong, Thinh Nguyen, Bella Bose
Abstract
This paper studies cross-domain lossy compression through the lens of minimum entropy coupling (MEC) with rate and classification constraints. In this setting, an encoder observes samples from a degraded source domain, while the decoder is required to generate outputs following a prescribed target distribution and to preserve information relevant to a downstream classification task. Motivated by logarithmic-loss distortion, we adopt an information-based objective that maximizes the coupling strength between the source and reconstruction, rather than minimizing a sample-wise distortion. Under common randomness, we formulate a rate-constrained MEC problem (MEC-B) and show that the intermediate representation can be removed without loss of optimality, yielding an equivalent deterministic coupling formulation. For Bernoulli sources, closed-form expressions are derived with and without classification constraints. In addition, we implement a neural restoration framework using quantization, entropy modeling, distribution matching, and classification regularization. Experiments on MNIST super-resolution and SVHN denoising show that increasing the available rate improves classification accuracy and yields more informative reconstructions.