How generative compression systems transfer semantic information across tasks
Semantic Transferability of Generative Rate-Distortion-Perception Systems
Information Theory
Summary
Compression systems try to shrink data while keeping what matters most, like the meaning behind images or sounds. This paper studies how parts of such a system, built for one task, can be reused for a different but related task. The authors examine three ways to reuse parts—from only reusing the decoder to keeping the whole compression model. They provide exact conditions and mathematical limits for when and how semantic information can be transferred with minimal loss.
What this means in practice
- •For data compression engineers: Reuse parts of a trained semantic compression system to support new generation tasks without redesigning the whole model, saving development effort.
- •For machine learning system designers: Design modular generative compression pipelines that can transfer semantic representations efficiently across related tasks.
A theory result. No direct application yet.
Authors
Jinwan Jeon, Seoyeon Hwang, Sung Hoon Lim
Abstract
Generative lossy compression combines a rate-limited representation with a stochastic decoder that preserves source-dependent information while generating unresolved detail. We ask what remains reusable when a rate-distortion-perception (RDP) system selected for one operating point is transferred to a prescribed conditional-generation task. We distinguish the single-letter RDP factorization from its block-code implementation and compare Full Redesign with three increasingly restrictive transfer strategies. Renderer-Only Transfer retains only the renderer and redesigns semantic steering. Representation-Induced Transfer additionally retains the semantic latent law induced by the original representation. Full-Model Transfer retains a compatible RDP code, marginalizing over the task conditional law to form its semantic encoder. For finite alphabets, we characterize Renderer-Only Transfer's exact compression-common-randomness rate region, renderer-hull controllability condition, and compression-rate penalty relative to Full Redesign. Under unlimited common randomness, this penalty is the minimum residual semantic information $I(S;U|X)$ forced by the inherited renderer. Representation-Induced Transfer is exact if and only if the semantic condition depends on the source only through the recurrent-class label $C$ of the end-to-end RDP test channel. Every compatible RDP code reused under Full-Model Transfer preserves $C$ and supports every task satisfying $X-C-S$. Any additionally transferable within-class semantic dependence must satisfy $I(S;X|C)\le R_0-H(C)$, where $R_0$ is the inherited compression rate. The bound is exact when $R_0=H(C)$. A scalar Gaussian specialization yields a renderer-noise controllability threshold, a closed-form transfer penalty, and examples of identical Gaussian RDP test channels with different Renderer-Only Transfer capabilities.