Scene manifold modeling improves quality scores for 3D view rendering
From Explicit References to Scene Manifolds: Distributional Fidelity and Realism for Radiance Field Quality Assessment
Computer Vision and Pattern Recognition
Summary
When creating new views of 3D scenes from digital models, errors and odd visual glitches can appear. Existing methods to check image quality compare each view directly to a reference image, which is hard when views come from different angles or positions. The paper introduces SCODA, a method that checks quality by understanding the overall scene instead of comparing individual images. It models the scene using multiple example views, then measures how much new views fit into this model and look realistic. Their tests show SCODA agrees well with human opinions and works across different 3D rendering techniques.
What this means in practice
- •For 3d graphics engineers: Assess the visual quality of generated 3D views without needing perfectly aligned reference images.
- •For virtual reality developers: Improve evaluation of rendering quality for scenes viewed from arbitrary trajectories in VR environments.
Authors
Saeed Mahmoudpour, Gi-Mun Um, Hyon-Gon Choo, Peter Schelkens
Abstract
Radiance field representations such as 3D Gaussian Splatting (3DGS) enable high-quality novel view synthesis but can introduce complex, view-dependent artifacts from reconstruction, rendering, and compression. Reliable perceptual quality assessment (QA) is thus essential for evaluating rendered views and guiding the design of perceptually faithful scene representations. Existing full-reference QA metrics require an aligned reference image, while recent cross-reference metrics relax this requirement by comparing a test view with non-aligned references. However, under wide-baseline radiance field settings, selecting a reliable nearby reference can be difficult, particularly when evaluating views along arbitrary trajectories and poses. We propose SCODA, a lightweight scene-conditioned objective QA method that shifts QA from explicit image-to-image comparison to scene-manifold modeling. High-quality observations of each scene are represented as a multivariate Gaussian distribution in deep feature space, producing a semantic fidelity score that measures deviation from the scene distribution. A weakly-supervised distortion-aware patch discriminator provides a complementary realism signal, and both cues are combined through an unsupervised bounded fusion strategy. Experiments on multiple benchmarks show strong agreement with human judgments and robust generalization across GS- and NeRF-generated views and trajectories. Code is publicly available at https://gitlab.com/saeedmp/scoda.