Physics guided model improves realistic object insertion in images
A Bridge from Paste to Place: Physics-Guided One-Step Image Composition via Object-Scene Interaction
Computer Vision and Pattern Recognition
Summary
Making an object look naturally part of a new photo is tricky because it needs to match lighting, shadows, and details perfectly. The authors developed a model called Paste2Place that uses physics principles to directly transform pasted objects into seamlessly blended images in one step. They created a big dataset with realistic examples to train and test their model. Their experiments show it does better and faster than previous methods at making objects look genuinely present in new scenes.
What this means in practice
- •For digital artists: Create more realistic composite images by automatically blending pasted objects with scene lighting and shadows in one step.
- •For advertising agencies: Produce realistic promotional images with objects seamlessly inserted into scenes efficiently, speeding up the creative process.
Authors
Hongzhen Li, Youwei Pan, Yingfang Zhu, Sen Zhu, Fengjie Zhu, Leilei Cao
Abstract
Realistic image composition must make a pasted object appear naturally present in a scene while preserving the fine details of both foreground and background. Reference-based insertion methods can change object pose and appearance, while foreground harmonization methods and shadow-specific synthesis methods each address only part of this task. An efficient, unified solution to this task remains an open challenge. We introduce Paste2Place, an end-to-end, physics-guided flow-based model that learns a direct, one-step latent bridge from pasted inputs to realistic composites, guided by explicit albedo-shading-residual decomposition and recomposition within the model. To support model training and evaluation, we develop a Blender rendering and validation pipeline to generate paired composites with physical annotations, and construct real-scene pairs from open-source object-removal data. This effort yields 86,273 training pairs and a 1,500-example benchmark. Extensive qualitative and quantitative experiments demonstrate that Paste2Place achieves substantial advantages in image composition quality and inference speed. Overall, our work provides a clear task definition and an effective solution for image composition, and fills a gap in large-scale, high-quality datasets and benchmarks for training and evaluating image composition.