Human preference based reward improves image cropping quality
CropReward: A Human-Preference Benchmark and Reward Model for Aesthetic Image Cropping
Computer Vision and Pattern Recognition
Summary
Cropping images well means cutting out distracting parts so the picture looks better. The usual ways to check if a crop is good don’t always match what people actually like. The authors created a big set of images with pairs showing which crops people prefer, then made a new way to score crops using a smart language and vision model. Their new score matches human taste better and helps build cropping tools that make nicer pictures.
What this means in practice
- •For photo editing software developers: Integrate a human-aligned scoring model to automatically select more appealing image crops in photo editing tools.$Commercial implications: This enables photo editing apps to offer superior auto-cropping features that better match user preferences, enhancing user satisfaction.
- •For digital marketing teams: Use the reward model to generate visually attractive image crops for advertisements that better engage audiences on social media.
Authors
Xiangfei Sheng, Xiaofeng Pan, Leida Li, Qian Zhang, Biao Wang, Johan Wagemans, Weisi Lin
Abstract
Aesthetic image cropping (AIC) aims to improve image composition and aesthetic quality by removing extraneous or distracting content. Over the past decade, AIC methods have evolved from specialized vision models to multimodal large language models(MLLMs). Reliable evaluation that reflects human preferences is essential for benchmarking AIC methods and guiding model optimization. However, the current AIC evaluation relies largely on geometry-based metrics such as intersection over union (IoU), as well as image aesthetic assessment(IAA) models. Although inconsistencies with human preferences have long been observed, the robustness and reliability of these evaluation criteria remain underexplored. To bridge this gap, we first construct CropBench, a large-scale benchmark comprising 106,995 human preference pairs for 1,019 images, with each candidate set containing the original image and candidate crops produced by 14 AIC methods. A systematic study on CropBench reveals that the existing criteria, ranging from IoU to off-the-shelf IAA assessment models, only exhibit limited alignment with human preferences. Building upon CropBench, we further propose CropReward, an MLLM-based reward model that evaluates a candidate crop by comparing with the original uncropped image and outputs a score indicating explicit aesthetic improvement or degradation relative to the original. In addition, we employ CropReward to construct CropReward-300k, the largest AIC dataset to date, and further develop CropAlign, a preference-aligned AIC model trained solely with CropReward supervision, without human-annotated crop boxes. Experimental results demonstrate that CropReward aligns with human preferences significantly better than existing criteria. CropAlign also surpasses the state-of-the-art AIC methods, demonstrating the advantage of CropReward in guiding AIC model optimization.