Adaptive bidirectional learning improves 3D scene representations
RouteBridge: Reliability-Routed Bidirectional Distillation Between Neural Radiance Fields and 3D Gaussian Splatting
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
This paper deals with two popular ways to digitally represent 3D scenes: neural radiance fields (NeRFs) and 3D Gaussian splatting (3DGS). Each method has strengths and weaknesses, and previous attempts to combine them used one as the teacher for the other across the whole scene. The authors created RouteBridge, a system that decides the best teaching direction for each view ray, improving image quality by using a combination of visual errors and geometric clues. This flexible approach leads to better 3D scene reconstructions than using either method alone or previous fixed-teacher approaches.
What this means in practice
- •For 3d graphics developers: Create more accurate and visually detailed 3D scene models by combining neural and Gaussian-based methods with adaptive bidirectional knowledge transfer.
- •For virtual reality teams: Improve quality and consistency of 3D environments by routing learning direction dynamically to handle local reconstruction errors in scene encoding.
Authors
YuanHang Wang, Xin Cao
Abstract
Neural radiance fields (NeRFs) and 3D Gaussian Splatting (3DGS) encode a scene with complementary inductive biases, but existing cross-representation distillation typically fixes one representation as teacher for the entire scene. A globally fixed teacher can propagate local reconstruction errors. We present RouteBridge, a bidirectional framework that selects the teaching direction for each ray. Its reliability estimator combines photometric residuals with representation-specific geometric evidence and routes supervision from NeRF to 3DGS, from 3DGS to NeRF, or abstains. A renderer-independent interface transfers color, opacity, and normalized depth without shared features or point correspondence. On mip-NeRF 360, the NeRF and 3DGS exports reach 28.56 and 28.77 dB, respectively. The 3DGS export improves over 3DGS by 1.56 dB and over NeRF-GS by 0.45 dB while reducing LPIPS to 0.207. On static three-view DTU, RouteBridge obtains 21.12 dB. Ablations show that both adaptive routing and geometric ray targets contribute to the improvement.