TISC: A Text-Driven Image Semantic Communication System for Faithful Reconstruction
2026-08-17 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionNetworking and Internet Architecture
AI summaryⓘ
The authors studied a way to send images by turning them into text descriptions and then recreating the images from that text using special computer models. They found two main problems: the text descriptions sometimes miss important details about objects in the image, and the image recreation can vary a lot depending on random settings. To fix this, they created a method called TISC that breaks down the image description into detailed parts about the scene and objects, and also chooses better starting points for recreating images. Their tests showed that this approach helps make the recreated images more accurate and true to the original.
semantic communicationimage-to-text (I2T)text-to-image (T2I)diffusion-based generative modelssemantic extractionsemantic reconstructionTree-Structured Attribute Semantic Extraction (TSASE)Initial Noise Optimization (INO)semantic faithfulnessimage reconstruction
Authors
Feifan Zhang, Yuyang Du, Xiaoyan Liu, Soung Chang Liew
Abstract
Generative image semantic communication converts an image into a text description and then performs text-to-image reconstruction at the receiver via diffusion-based generative models. This paradigm has attracted broad attention due to its extremely low bandwidth cost. However, existing methods still face two critical bottlenecks across image-to-text (I2T) semantic extraction at the transmitter and text-to-image (T2I) semantic reconstruction at the receiver: (i) semantic loss and distortion in I2T, where holistic image descriptions may omit fine-grained object attributes and spatial-position information, causing the generated text to deviate from the original image semantics; and (ii) insufficient semantic faithfulness in T2I, where even with the same semantically faithful text description, different initial noise settings may lead diffusion-based reconstruction to produce images with different levels of semantic consistency with the original image. These issues jointly limit the semantic faithfulness of image reconstruction. To address them, we propose TISC, a text-driven image semantic communication framework tailored for faithful reconstruction. TISC incorporates two key designs: (1) Tree-Structured Attribute Semantic Extraction (TSASE), which decomposes semantic extraction into global scene, background, and object-level attribute descriptions, covering spatial position, shape/pose, color, material, and other physical attributes for each detected object; and (2) an Initial Noise Optimization (INO) mechanism, which selects an initial noise seed at the transmitter according to a comprehensive similarity score that jointly considers visual and semantic consistency. Experiments on multiple datasets show that TSASE improves object-position recovery and semantic description faithfulness, while the INO parameter study supports the adopted configuration for noise selection.