OmniShowcase generates consistent 3D subject videos from multi-view images
OmniShowcase: Omnidirectional Subject-Consistent Video Generation from Multi-View Images
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Creating videos of a person or object from just a few pictures is hard when some parts can't be seen in the original photos. The researchers made a new way to produce videos that show the subject consistently from all angles using multiple pictures and 3D shape information. They built a system that uses computer-generated data and surface details to help the video match the true 3D form, even when rotating fully around the subject. Their tests show better identity preservation and 3D consistency compared to earlier methods.
What this means in practice
- •For computer graphics developers: Create realistic 3D-consistent subject videos from a few images for media and entertainment projects.
- •For augmented reality software teams: Use multi-view geometry-conditioned videos to improve subject realism in AR experiences with changing viewpoints.
Authors
Junyeong Ahn, Geonwoo Kim, Kinam Kim, Gihwi Kim, Hoonjin Jung, Hyojin Jang, Jaegul Choo
Abstract
Generating videos of a specific subject from a few reference images remains challenging when the camera reveals surfaces that are not visible in the input. Early subject-to-video (S2V) methods condition on a single reference image and must therefore infer the subject's unseen appearance and geometry as the viewpoint changes. To address this limitation, recent methods extend S2V to multiple reference views, providing more information about the subject from different viewpoints. However, they do not explicitly specify the 3D structure that connects these observations. In addition, existing training data do not guarantee that viewpoint changes preserve the same underlying 3D subject. We present OmniShowcase, built on two components: (i) a fully automated data synthesis pipeline that produces training videos along diverse camera trajectories, with viewpoint variation driven by underlying subject geometry, even spanning full 360-degree orbits; and (ii) a geometry-conditioning method that provides the video model with a turntable of surface-normal maps reconstructed from the reference views without requiring temporal alignment with the generated frames. We construct Showcase-4K, a corpus of 3D-asset-based videos covering twelve camera motions across 88 trajectory configurations with reconstruction-derived surface-normal conditioning, and ShowcaseBench, a held-out evaluation set with the source meshes and camera parameters used to render each video. Experiments show state-of-the-art subject identity and 3D consistency while maintaining competitive video quality and motion. Project page: justin4ai.github.io/OmniShowcase