Vehicle shape and size recovered from limited images using priors
Leveraging Visual and Geometric Priors for Metric-scale and Complete Vehicle Gaussian Reconstruction from Limited Views
Computer Vision and Pattern Recognition
Summary
It is difficult to create accurate 3D models of vehicles from just a few photos taken from one side because the scale is unclear and the other side is missing. The authors use visual clues from a foundation model to start the shape and size, then apply a special method that copies the seen side to the unseen side, since most vehicles are symmetrical. This approach makes it possible to build a complete and correctly sized 3D vehicle model even from just one viewpoint. Their tests show this method works better than others in creating full and precise vehicle shapes.
What this means in practice
- •For traffic simulation developers: Generate complete and accurately scaled 3D vehicle models from limited onboard camera views to simulate rare traffic scenarios.
- •For video game artists: Create precise and symmetrical 3D vehicle assets from sparse photos for use in virtual environments and video games.
Authors
Jinyu Miao, Jiusi Li, Yifei He, Miao Long, Kun Jiang, Mengmeng Yang, Diange Yang
Abstract
High-fidelity vehicle assets are essential for controllable traffic scene generation, particularly for synthesizing rare and safety-critical long-tail scenarios. However, reconstructing a reusable vehicle representation from in-the-wild onboard images remains challenging for two reasons. First, image-to-3D generation methods generally produce models without reliable metric scale. Second, onboard cameras usually observe only one side of a target vehicle, making conventional multi-view reconstruction incomplete on unobserved regions. To solve these problems, we propose a feed-forward vehicle asset reconstruction method, which leverages two complementary priors to reconstruct 3D Gaussian representations for vehicles using sparse one-sided observations. To achieve metric-scale reconstruction, a visual foundation model is first utilized to serve as a visual prior for Gaussian initialization. The Gaussian attributes are then estimated by a learnable encoder-decoder module. A symmetry-aware cloning strategy is presented to complete the unobserved side directly in Gaussian space, which exploits the bilateral structure of vehicles as a geometric prior. Experiments on the public dataset demonstrate that the proposed method significantly outperforms existing approaches in both vehicle asset completeness and geometric accuracy.