Cross-Branch Conflict as a Shield: Safeguarding Facial Identities in Unified Multimodal Image Editing
Computer Vision and Pattern RecognitionComputation and LanguageCryptography and Security
Summary
The authors studied how advanced image editing models, which understand and generate images using two different methods, can be tricked to prevent unauthorized changes to personal photos. They found that messing with just one method isn’t enough to stop the model from recognizing the original image because the two methods support each other. To fix this, they created a new approach called CCS that confuses both methods at the same time and makes them disagree, so the model can't reliably edit the person's identity. Their tests show this method works better than older ones at protecting people’s images.
Authors
Weiwei Tan, Junxian Li, Rui Wang, Zhenhua Xu, Yanjun Zhang, Yu Leo Zhang
Abstract
Unified multimodal models (UMMs) have recently demonstrated powerful instruction-based image editing capabilities, but they also raise serious concerns about unauthorized manipulation of personal portraits. Existing adversarial protection methods are mainly designed for either visual understanding or image generation models and often become ineffective when transferred to UMMs, which process an image through multiple complementary visual pathways. In this work, we first conduct a feature-level analysis of unified image editing. We observe that the ViT-based understanding branch and the VAE-based generation branch exhibit non-trivial structural agreement for the same input image. Although perturbing an individual branch can reduce this agreement and induce intermediate hidden-state deviations, such effects are asymmetric and gradually attenuated during multimodal fusion and generation. These observations reveal that single-branch feature distortion is insufficient for consistently disrupting unified image editing. Motivated by this finding, we propose CCS, a unified adversarial protection framework that jointly drives the ViT and VAE representations away from their clean counterparts while explicitly disrupting their cross-branch compatibility through linear CKA. By simultaneously removing stable information from both visual pathways and creating incompatible visual contexts, CCS prevents the UMM from recovering reliable identity information during editing. Extensive experiments demonstrate that CCS consistently outperforms existing protection methods in suppressing identity-preserving edits.