Camera intrinsics guide test-time 3D geometry adaptation for better accuracy

IntriAdapt: Generalizable Intrinsic-Guided Test-Time Adaptation for Metric Geometry Estimation

Computer Vision and Pattern Recognition

Summary

Figuring out accurate 3D shapes from images is tricky because cameras distort what they see. The authors found a way to improve 3D shape guesses by using known camera details during testing without needing extra depth sensors. They created a method, IntriAdapt, that checks if the shape fits the camera’s view rules and tweaks its guesses to match. This approach works well even when seeing videos from the same camera and performs better than previous methods on many tests.

What this means in practice

  • For computer vision engineers: Improve 3D shape predictions during live image processing without additional depth hardware by adapting models using camera-specific details.
  • For augmented reality developers: Enhance metric accuracy of 3D reconstructions from camera footage in AR applications by applying model adaptation guided by known camera intrinsics.

Authors

Jiaxing Chen, Zizheng Yan, Jingyu Zhuang, Jinwei Chen, Shuguang Cui, Guanying Chen, Qingnan Fan

Abstract

Recovering metric 3D geometry from images is fundamentally ill-posed due to perspective projection ambiguity. Current geometry models incorporate supplied or predicted intrinsics during inference, but do not use them to supervise model adaptation. To refine these predictions, test-time adaptation (TTA) typically relies on external depth measurements. In this paper, we instead use camera intrinsics as an output-side geometric verifier. Comparing the point cloud's viewing rays with those defined by the known intrinsics yields a direct measure of geometric inconsistency. We propose \textit{IntriAdapt}, a TTA framework that minimizes this closed-form geometric residual. By optimizing a lightweight LoRA adapter on a frozen geometry foundation model, our method aligns the prediction with the specific camera model without depth sensors or external priors. We further extend this approach to video sequences, demonstrating that adapting a short sequence generalizes to other sequences captured by the same camera. The adapted model processes held-out images in a feed-forward pass, without repeating optimization for each frame. Experiments on ten single-image and seven sequence benchmarks across multiple base models show that \textit{IntriAdapt} consistently outperforms both unadapted and oracle-intrinsics baselines in metric accuracy and relative geometry.