Streaming 4D capture tracks fast outdoor subjects from sparse cameras
Racing in Volume with Flow Ensembles
Computer Vision and Pattern Recognition
Summary
Capturing 3D videos of fast-moving objects outdoors using a few fixed cameras is very challenging. The authors built FastFlowGS, a system that combines different ways to track motion and reconstruct shapes in real time. They also created Monaco4D, a virtual benchmark with realistic racing scenes to test such systems. FastFlowGS outperforms previous methods in quality and speed on existing and new datasets. This advances real-time 4D capture for fast outdoor events from limited viewpoints.
What this means in practice
- •For sports broadcast teams: Create real-time 3D visualizations of fast-moving athletes using few fixed trackside cameras for enhanced viewing experiences.$Commercial implications: Enables production and sale of immersive 4D sports coverage products that improve spectator engagement.
- •For autonomous vehicle developers: Improve perception models by incorporating high-speed dynamic scene reconstructions from sparse external cameras monitoring traffic.
Authors
Saswat Subhajyoti Mallick, Riu Cherdchusakulchai, Marc Ruiz Olle, Albert Mosella-Montoro, Jose Ribeiro-Gomes, Francisco Vicente Carrasco, Fernando De la Torre
Abstract
Streaming 4D reconstruction has been demonstrated only indoors, on dense camera rigs surrounding subjects that move at human pace. Outdoor 4D reconstruction exists but relies either on cameras mounted on the moving vehicle itself, or on limited-coverage arrays observing quasi-static subjects offline. The case that actually matters for spectators is a fast-moving subject, watched from a sparse ring of allocentric cameras, streaming. No method targets this, and no benchmark exists to evaluate one. To this end, we introduce FastFlowGS, a streaming 4D Gaussian Splatting method for reconstructing fast-moving subjects from a small set of fixed external cameras, and Monaco4D, a photorealistic Unreal Engine 5 benchmark for high-speed outdoor reconstruction. FastFlowGS fuses sparse matches, semi-dense tracks, and dense optical flow by lifting each signal to 3D with geometric uncertainty and combining them through a Kalman-style temporal update. Monaco4D provides Formula 1 sequences under varied illumination from trackside, onboard, and drone viewpoints with dense ground truth. On CMU-Panoptic, FastFlowGS exceeds the strongest baseline by 12.6% VMAF at 35% greater efficiency. On Monaco4D, where existing streaming methods degrade severely, it improves dynamic-region PSNR by up to 18.6% with 28.3% lower per-frame optimization time. Dataset and additional details can be found at https://humansensinglab.github.io/monaco4d/.