Visual geometry transformer scales high definition 3D reconstruction efficiently
ReDT: Resolution-Decoupled Visual Geometry Transformer
Computer Vision and Pattern Recognition
Summary
High-resolution 3D reconstruction from multiple images usually requires heavy computation and struggles with quality as resolution increases. The authors propose ReDT, a new transformer design that processes images by converting them to a fixed-size summary before analyzing them, allowing it to handle high-definition inputs faster and use less memory. This method improves 3D reconstruction quality and runs much faster compared to previous approaches. Their tests show strong performance gains on various datasets including very high resolutions.
What this means in practice
- •For 3d imaging developers: Build faster and more memory-efficient 3D reconstruction tools that handle high-resolution input images without quality loss.
- •For visual effects studios: Produce higher quality 3D models from multi-view HD footage while reducing rendering time and hardware requirements.$Commercial implications: Enables faster and more cost-effective production of 3D assets for films and games, improving workflows and reducing hardware costs.
Authors
Abteen Arab, Guile Wu, Dongfeng Bai
Abstract
We introduce ReDT, a feed-forward visual geometry transformer designed to scale multi-view reconstruction to high-definition (HD) inputs. Existing visual geometry transformers are typically trained at fixed, standard definition (SD) resolutions and exhibit degraded reconstruction quality and poor computational scaling to high input resolutions. While recent efficient architectures reduce the complexity of inter-frame attention, expensive intra-frame and encoder attention remain largely unaddressed, making HD reconstruction impractical with existing architectures. ReDT addresses both limitations with a resolution-aware architecture designed to scale to high-resolution inputs. The key idea of ReDT is to first map each view to a fixed-size set of basis tokens, compute correspondences in this compact constant-size basis space, and then project the outputs back to the original resolution for dense prediction. By decoupling the internal representation from input resolution during attention, ReDT substantially reduces the computational cost of high-resolution 3D reconstruction and improves model performance at all resolutions. Extensive experiments on multiple datasets spanning SD to FHD (2K) show that ReDT outperforms state-of-the-art visual geometry transformers in both reconstruction quality and computational efficiency. For instance, on the ETH3D (2K) dataset, ReDT achieves a $16\times$ speedup, a $34\%$ reduction in peak GPU memory usage, and a $33\%$ reduction in chamfer distance over VGGT-$Ω$. \href{https://redt.github.io}{Project Website.}