Diffusion model improves same person face replacement in photos
PicturePerfect: Diffusion-based Same-identity Facial Replacement
Computer Vision and Pattern Recognition
Summary
Replacing a person's face in a photo with another image of the same person is tricky, especially when the photos have different lighting or expressions. The authors developed a method that uses a special type of AI called a diffusion model to swap faces in pictures more naturally and realistically. They built a big dataset of photos showing the same person with different faces to help train and test their method. Their results look better and keep the person's identity more consistent than previous techniques.
What this means in practice
- •For photo editing software developers: Integrate advanced face replacement that maintains identity consistency for portrait photo retouching and enhancement.$Commercial implications: Enables face replacement features to be sold as part of commercial photo editing tools that deliver more realistic and identity-preserving edits.
- •For film and video production teams: Create seamless same-person facial replacements in footage for visual effects and post-production purposes.
Authors
Manik Dhingra, Yashesh Savani, Mahesh Reddy, Antoine Mercier, Jiahe Liu, Guillaume Berger
Abstract
With the growing success of diffusion models in computer vision, we explore their potential for same-identity facial replacement in photographs. Specifically, we propose a diffusion-based method, built on top of a pre-trained text-to-image model, that takes as input a portrait image of a person and a second reference image of the same individual, potentially captured under different conditions. The goal is to seamlessly replace the input face with the reference face, while keeping the background intact. Surprisingly, despite the clear real-world utility of this task, no recently published work has directly addressed face replacement in this specific setting. To support this goal, we construct a large dataset of image pairs depicting the same person under varying facial expressions and poses. Experimental results demonstrate that our approach produces more realistic and identity-consistent results than existing face reenactment models.