Implicit neural fields improve view synthesis without pose data
IRIS: Implicit Rendering Matters for Pose-Free Novel View Synthesis
Computer Vision and Pattern Recognition
Summary
Synthesizing new views of a scene without knowing camera positions is tough because the system must learn both the 3D shape and camera poses at once. The authors propose IRIS, a method that combines the strengths of flexible but weakly grounded models with those having strong geometry but complex setups. IRIS represents scenes as latent neural fields that use features projected from known views to create new images from new viewpoints. This approach balances flexibility and geometric structure to produce accurate novel views and camera poses without supervision.
What this means in practice
- •For augmented reality developers: Generate realistic new views of scenes without needing precise camera pose inputs, simplifying content creation for AR experiences.
- •For robotic vision teams: Enable robots to build and visualize 3D scene views from uncalibrated multi-camera inputs, improving spatial awareness with less manual setup.
Authors
Wenyu Li, Sidun Liu, Peng Qiao, Yong Dou, Tongrui Hu
Abstract
Novel view synthesis from unposed multi-view images remains challenging, as the model must jointly learn scene representations and camera parameters without pose supervision. Existing approaches largely fall into two extremes: implicit latent-space rendering is flexible and easy to optimize, but often yields weakly grounded camera estimation; explicit 3D representations provide stronger geometric grounding, but introduce heavier parameterization and more fragile optimization. In this paper, we present IRIS, a fully self-supervised framework that provides a practical middle ground between these two paradigms. Instead of decoding free latent tokens or reconstructing fully explicit 3D primitives, IRIS represents the scene as a latent neural field and renders novel views by querying this field under self-predicted cameras. Specifically, projected features from reference views are aggregated at sampled 3D points to form point-wise latent features, which are then composed along target rays for rendering. This design preserves the flexibility and optimization stability of implicit modeling, while introducing stronger geometric structure than unconstrained latent rendering. Extensive experiments show that IRIS achieves strong novel view synthesis quality with competitive pose accuracy under fully self-supervised learning. Our project page: https://leo-frank.github.io/IRIS