Image editing method finds better ways to change pictures realistically
Grounded Exploration for Open-Ended Image Editing
Computer Vision and Pattern Recognition
Summary
Sometimes when people ask a computer to change a picture in an open-ended way, many changes can be done and just letting the computer pick randomly can create weird or unrealistic edits. The authors propose a two-step approach where the computer first figures out different meaningful areas or ideas to change and then makes specific instructions to edit the picture accordingly. This method creates more varied and meaningful changes that still make sense, compared to just asking for diverse changes directly. Their tests show this makes edited images more interesting and grounded in real possibilities.
What this means in practice
- •For graphics software developers: Create image editing tools that suggest diverse yet plausible edits for open-ended user requests.$Commercial implications: Improves usability and creativity in photo editing applications by offering richer, realistic editing options to consumers and professionals.
- •For game environment designers: Generate varied but coherent changes to scene visuals based on flexible edit goals to speed up environment design.
Authors
Jieyu Zhou, Mehdi Ben Ayed, Shubham Gupta, Zhenyu Tan, Jiajing Xu, Dmitry Kislyuk, Eric Tzeng, Christopher J. MacLellan, Charles Rosenberg
Abstract
Open-ended image-editing requests, such as ``decorate this room,'' admit multiple valid realizations. Direct sampling may produce ungrounded variation, whereas explicit diversity prompting often produces superficial differences. We study how to ground these requests without prematurely eliminating their legitimate ambiguity. We introduce a two-stage framework that first identifies diverse, feasible edit scopes and then realizes them as executable edit instructions. Across 300 open-ended image-editing tasks, our approach achieves 24.6% higher mean semantic dispersion and 17.1% higher mean groundedness than direct diversity prompting, while producing more valid and distinct realizations. These results show that structuring feasible edit directions before realization supports more useful grounded exploration.