Shadow removal improves with new diverse training data set
After a Decade: Bringing Shadow Removal into the Real World with Agentic Training Data
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Shadow removal software works well on standard tests but struggles with real-world photos because it lacks varied examples to learn from. The authors created a new approach to build a large collection of real photos showing shadows and their shadow-free versions by combining physics knowledge, error checks, and automatic fixes. This dataset helps models learn better and handle different scenes, faces, and satellite images more accurately. Training on this new data reduced color errors significantly compared to previous methods.
What this means in practice
- •For photo editing software developers: Improve shadow removal features in photo editing tools by using the new dataset covering diverse real-world scenes.$Commercial implications: Enables more reliable shadow removal in consumer photo apps, enhancing user experience and product competitiveness.
- •For remote sensing analysts: Enhance shadow correction in satellite imagery processing to improve analysis accuracy using the dataset's remote sensing images.
Authors
Shilin Hu, Jingyi Xu, Dimitris Samaras, Hieu Le
Abstract
Shadow removal looks nearly solved on established benchmarks, yet remains brittle in the real world. Models have advanced; the paired training data they rely on have barely changed in nearly a decade. The reason is simple: obtaining a shadow-free target requires removing the occluder while keeping the scene, camera, and illumination otherwise unchanged, making diverse paired data difficult to capture. Meanwhile, large shadow detection datasets already contain diverse real-world images and masks, but no shadow-free targets. To turn this abundant but incomplete data into paired supervision, we propose an offline agentic workflow combining physics-motivated generation, failure detection, feedback-driven retry, candidate selection, and deterministic correction. Using this workflow, we construct AgenticShadow, a dataset of 17,138 image-mask-target triplets spanning general scenes, faces, and remote sensing. Our construction workflow reduces Color Distribution Difference by 50.5% over previous shadow removal work, while training existing shadow removal models on AgenticShadow reduces cross-domain LAB RMSE by 19.7-37.5%.