Drift removes diffusion watermarks by altering generation paths
DRIFT: Removing Diffusion Watermarks by Deflecting the Generative Trajectory
Cryptography and SecurityMachine Learning
Summary
Diffusion watermarks hide secret signals in images made by AI to prove who created them, but these signals can be hard to remove without ruining the image. The authors found a common weakness in how these watermarks rely on retracing the image creation steps. They created DRIFT, a method that mixes up these steps using random noise and partial redoing, making it hard for watermarks to survive. DRIFT works well on many types of watermarks and keeps image quality high without needing to know secret details or test each image separately.
What this means in practice
- •For image forensic analysts: Detect weaknesses in current diffusion watermarking methods by simulating removal attacks without knowledge of keys or verifiers.
- •For digital content moderators: Use DRIFT to clean images of hidden watermarks that affect authenticity checks while preserving image quality for further moderation.
Authors
Rui Bao, Zheng Gao, Xiaoyu Li, Xiaoyan Feng, Yang Song, Jiaojiao Jiang
Abstract
Diffusion watermarking embeds verifiable signals into the generative process and commonly verifies them by recovering trajectory-dependent evidence, making the marks robust to conventional pixel-space distortions. Existing removal attacks either regenerate along deterministic trajectories, which often preserve the watermark-bearing latent structure, or optimize every image separately. We identify the reliance on a recoverable generative trajectory as a common attack surface among the schemes we study. Based on this observation, we propose DRIFT, a black-box attack that combines partial forward diffusion with stochastic reverse resampling. Forward re-noising limits source information available to a fixed-depth recovery pipeline, while stochastic reversal supplies alternative noise-driven paths whose removal benefit we isolate through matched sampler comparisons. Adaptive DRIFT searches a selected ladder for each image's first verifier-rejected rung and refines fidelity while retaining only updates rejected by the same verifier. At fixed depth, we derive information-theoretic and Wasserstein source-dependence bounds; under realized-ladder monotonicity, the first rejected rung is least distorted among rejected rungs on that ladder, and verifier-gated refinement preserves rejection. Across nine watermarks spanning three paradigms, DRIFT achieves 98-100% attack success and the best image quality among the compared attacks, without secret keys, verifier internals, or per-image gradient optimization.