SkillIR improves image restoration by evolving scene-aware skills
SkillIR: Evolving Scene-Aware Skills for Agentic Image Restoration
Computer Vision and Pattern Recognition
Summary
Fixing images damaged in complicated ways is tough because each repair step can change what needs to be fixed next. The authors developed SkillIR, which learns how each repair tool works in different situations and adjusts repairs one step at a time based on what happened before. This method helps pick better next steps by checking results before moving on, making image fixes more accurate. SkillIR also remembers what it learned to get better with practice over time.
What this means in practice
- •For photo editing software developers: Enhance photo editors by integrating adaptive tools that plan image repairs step-by-step for images with multiple damage types.
- •For digital archivists: Improve automated restoration of degraded historical images by using evolving skill-based methods that better handle complex damage patterns.
Authors
Jie Shao, Shengkai Hu, Xu Zhang, Beihang Song, Yongcheng Jing, Xu Wu, Jun Wan
Abstract
This paper studies agentic image restoration, in which multimodal agents coordinate specialized restoration tools to recover images affected by complex degradations. Existing restoration agents often derive complete tool-use plans from the original degraded image or retrieve previously successful trajectories, providing limited support for adapting individual actions to evolving intermediate restoration states. We find that accepted tool executions can change the residual degradation state and, consequently, the applicability of subsequent tools. To address this issue, we propose SkillIR, a skill-guided framework that represents restoration experience as degradation-centered action evidence rather than complete tool-use trajectories. SkillIR consolidates context-dependent action outcomes into scene-aware restoration skills that characterize applicable conditions, expected effects, and attributable failure cases. Instead of prescribing a complete restoration plan, the retrieved skills guide one bounded action at a time within a verified residual-state loop: each tool output is treated as a candidate, committed only after transition verification, and followed by reassessment of the active residual degradations. After each rollout, the resulting evidence is used to create, refine, or patch dynamic skills, enabling accumulated restoration experience to improve decision-making for subsequent inputs. Experiments on synthetic and real-world multi-degradation datasets demonstrate that SkillIR improves restoration quality and enables more reliable and effective tool use.