Semantic image communication method resists strong adversarial attacks
TwinViT-DeepJSCC: Adversarially Robust Semantic Image Communication
Cryptography and Security
Summary
Semantic communication systems send important image details over the air but can be fooled by small, smart changes called adversarial attacks that mess up these details. The authors create TwinViT-DeepJSCC, which uses two different deep learning models working together to protect image information before sending it. Their method also cleans up any mistakes it finds after receiving the signal, without needing to know details about the attack. Tests show that this approach clearly improves image quality and classification accuracy compared to other methods when signals face different types of interference and attacks.
What this means in practice
- •For wireless communication engineers: Design image transmission systems that maintain high quality despite intentional or natural interference on wireless channels.
- •For security-focused network operators: Improve resilience of image communications against stealthy attacks aiming to disrupt or degrade image data during transmission.
Authors
Maedeh Fallahreyhani, Paeiz Azmi, Nader Mokari, M. Reza Abedi, Melike Erol-Kantarci, Eduard A. Jorswieck
Abstract
Learning-based semantic communication is vulnerable to adversarial perturbations introduced before semantic encoding or over wireless channels. This paper proposes TwinViT-DeepJSCC, a preventive-corrective semantic image transceiver operating under a fixed channel-use budget. Two Vision Transformer (ViT)-based deep joint source-channel coding (DeepJSCC) branches learn complementary latent representations protected by sensitivity-aware masking. At the receiver, confidence-aware fusion, blind corruption-severity estimation, and signal-to-noise ratio (SNR)-severity-conditioned denoising diffusion implicit model (DDIM) purification mitigate residual corruption without requiring attack metadata. Experiments on the Canadian Institute for Advanced Research 100-class (CIFAR-100) dataset consider fast gradient sign method (FGSM), projected gradient descent (PGD), natural evolution strategies (NES), and Carlini-Wagner (CW) source-domain attacks, as well as random jamming and channel-aware adversarial waveforms over additive white Gaussian noise (AWGN) and block-flat Rayleigh fading. Under matched channel-use and attack budgets, TwinViT-DeepJSCC achieves maximum peak signal-to-noise ratio (PSNR) gains of approximately 9.5 dB under 20-step PGD and 10.8 dB under channel-aware waveform attacks over block-flat Rayleigh fading. Under PGD, it also improves Top-1 accuracy by up to approximately 38 percentage points over the undefended baseline and 13 percentage points over the strongest competing defense. Ablation results confirm the complementary contributions of the proposed transmitter- and receiver-side mechanisms.