BlazeVGGT speeds up multi-view 3D reconstruction 13 times at 2000 frames
BlazeVGGT: 13x Faster VGGT at 2,000 Frames via Camera Token without Training
Computer Vision and Pattern Recognition
Summary
Processing many frames to reconstruct 3D scenes can be very slow because every frame is compared with all others. The authors noticed that early steps in the process capture local relationships, while later steps represent global structure. Using this insight, they created BlazeVGGT, which picks a smaller set of key frames for comparison without additional training. This method keeps all frames for detailed predictions but avoids redundant calculations, making the process much faster and even more accurate over long sequences.
What this means in practice
- •For 3d computer vision engineers: Accelerate 3D scene reconstruction workflows for very long video sequences by reducing redundant frame comparisons without losing detail.
- •For augmented reality developers: Enable faster camera pose estimation from multiple views to improve real-time tracking in AR applications with many input frames.
Authors
Shujun Yao, Junhui Lin, Congying Han, Tiande Guo, Ziwen Liu, Bonan Li
Abstract
Feed-forward visual geometry models, exemplified by VGGT, have demonstrated strong multi-view reconstruction capabilities through dense global attention. However, the resulting all-to-all cross-view computation scales quadratically with the number of input views, making long-sequence inference particularly expensive due to substantial frame-level redundancy among highly overlapping observations. We uncover a topology-first, geometry-later evolution in VGGT camera tokens: shallow-layer tokens reliably capture local view neighborhoods, whereas deeper-layer tokens progressively encode globally organized inter-view geometry. Motivated by this observation, we introduce BlazeVGGT, a training-free framework for structure-aware frame-level key--value sparsification. At each global-attention block, BlazeVGGT constructs a view-level similarity kernel from camera-token relations and employs greedy determinantal point process inference to select a compact set of mutually complementary views. The selected views constitute the key--value context, while all input frames are retained as queries. This asymmetric attention design preserves dense per-frame predictions while eliminating redundant cross-view computation, allowing our method to be directly integrated into pretrained VGGT as a plug-and-play module without parameter updates. Its advantages become increasingly pronounced with sequence length: at 2,000 frames, BlazeVGGT achieves approximately 13$\times$ faster inference than VGGT while reducing the Chamfer Distance from 0.41 to 0.32. Extensive experiments demonstrate a favorable accuracy--efficiency trade-off in point-cloud reconstruction and camera-pose estimation, compatibility with existing VGGT accelerators such as FastVGGT and LiteVGGT, and transferability to VGGT-based models including StreamVGGT and PAGE-4D. Codes are publicly available at https://github.com/MagiciannnY/BlazeVGGT.git.