PrismGPT improves photo editing with detailed region-aware plans
PrismGPT: Proxy-Guided Learning for Region-Aware Photo Editing with Self-Synthesized Reasoning
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Photo editing often needs both broad changes and specific tweaks in different parts of an image, but current tools only handle some of this automatically. The authors created PrismGPT, a system that looks at a single photo and decides how to edit both the whole image and its parts without relying on other commercial tools. They taught it simpler tasks first, like breaking down edits and ranking parts by how they look, helping it learn better editing decisions. Tests show PrismGPT matches or beats previous best tools while using far less training data.
What this means in practice
- •For photo editing software developers: Integrate region-aware editing plans directly from single images to improve automated photo retouching accuracy without external black-box tools.$Commercial implications: This enables photo editing software companies to offer advanced, data-efficient automated retouching features that deliver professional results with less training data.
- •For marketing teams for online retailers: Automatically enhance product photos with balanced global and local edits, improving image appeal to customers without manual adjustment.
Authors
Ke Zhao, Hue Nguyen, Abhijith Punnappurath, Zhongling Wang, Iqbal Mohomed, Michael S. Brown
Abstract
Professional photo finishing relies on both global adjustments and region-specific local edits guided by semantic masks, yet current automated methods handle this workflow only partially. We present PrismGPT, a Vision-Language Model (VLM) framework that produces structured, region-aware editing plans from a single input image without relying on commercial black-box tools. Training a VLM to simultaneously diagnose aesthetic deficiencies at both global and local levels while predicting precise editing parameters is challenging due to the vast combinatorial decision space. We address this through proxy-guided learning: two simpler proxy tasks -- operation decomposition and region-aware aesthetic ranking -- teach the foundational skills the model needs, while a competence-based dynamic scheduler automatically rebalances the multi-task training ratio, progressively shifting emphasis from the proxy tasks to the primary editing task as each skill is mastered. Crucially, all reasoning traces used for supervised fine-tuning are self-synthesized by the same base model, eliminating the need for a stronger external teacher. Experiments on MIT-Adobe FiveK and SPIRE, a new professionally retouched benchmark we introduce, show that PrismGPT achieves state-of-the-art results while using only ~6% of the training data compared to the previous best method.