WildIcon improves consistent animal image to video generation

Generating the Wild: Individual-Consistent Image-to-Video Generation for Wildlife

Computer Vision and Pattern Recognition

Summary

It can be hard to get many different pictures of the same wild animal to help identify it because animals move and look different from various angles. The authors address this by building a method called WildIcon that turns a single animal image into a video showing the animal in motion while keeping the animal’s unique details like fur patterns and spots consistent. They focus on capturing fine details linked to high-frequency image parts to make sure the animal’s identity stays clear. They also created a new dataset called WildlifeVid to help train and test models for this purpose. WildIcon produces videos that help identify individual animals and can aid in training systems that track animals by appearance.

What this means in practice

  • For wildlife conservation teams: Generate realistic videos from single animal images to increase individual animal data for monitoring and tracking without needing multiple real observations.
  • For computer vision engineers: Create identity-consistent animal video data from single images to augment training data for animal re-identification systems, improving tracking accuracy.

Authors

Yuzhuo Li, Di Zhao, Xinyu Zhang, Daniel Wilson, Yun Sing Koh

Abstract

Individual-level wildlife identification often suffers from data scarcity, as varying observations of the same animal under diverse poses, viewpoints, and motions are rarely available. Image-to-video (I2V) generation offers a promising way to mitigate this limitation by synthesizing additional observations from a single reference image. However, existing I2V models mainly emphasize global layout, semantics, and motion, and therefore often fail to preserve fine-grained local appearance cues that distinguish one wildlife individual from another, such as fur texture, stripe boundaries, spot configurations, and contour transitions. We observe that these identity-critical cues are closely related to high-frequency information. To address this challenge, we propose WildIcon, a high-frequency-guided I2V framework for wildlife individual consistency. Specifically, WildIcon introduces a frequency-aware identity encoding branch that extracts individual-specific high-frequency cues from the reference image. Combined with isolated foreground information, the resulting identity tokens are then injected into cross-attention blocks as identity conditioning. Building on a frozen backbone with lightweight identity adaptation, WildIcon preserves fine-grained identity cues visible in the reference image while retaining the motion controllability and semantic fidelity of the base I2V model. In addition, to support the training and evaluation of wildlife individual-consistent I2V, we construct WildlifeVid, a wildlife-centric video dataset with high-quality, temporally coherent clips and individual-level identity labels. Experiments on I2V generation and downstream animal re-identification (ReID) show that WildIcon achieves stronger individual consistency than existing baselines, and that its filtered outputs can serve as useful candidate training augmentations for downstream ReID.
View PDFOpen arXiv