Summary
Creating detailed 3D models of vehicles from a few photos taken from the side is hard because the scale might be wrong and parts of the vehicle are hidden. The authors developed a method that uses visual clues and knowledge about vehicle symmetry to build more accurate 3D models with the correct size. They start by estimating an initial shape using a visual model, then refine it using a learning system. To get the missing side of the vehicle, they copy the visible side’s shape, assuming cars are roughly symmetrical. Their approach makes more complete and accurate vehicle models than previous ways, which can help make better traffic scene simulations.
3D reconstructionGaussian representationmetric scalevisual foundation modelsymmetryencoder-decoder networkmulti-view reconstructiononboard camerastraffic simulationlong-tail scenarios
Authors
Jinyu Miao, Jiusi Li, Yifei He, Miao Long, Kun Jiang, Mengmeng Yang, Diange Yang
Abstract
High-fidelity vehicle assets are essential for controllable traffic scene generation, particularly for synthesizing rare and safety-critical long-tail scenarios. However, reconstructing a reusable vehicle representation from in-the-wild onboard images remains challenging for two reasons. First, image-to-3D generation methods generally produce models without reliable metric scale. Second, onboard cameras usually observe only one side of a target vehicle, making conventional multi-view reconstruction incomplete on unobserved regions. To solve these problems, we propose a feed-forward vehicle asset reconstruction method, which leverages two complementary priors to reconstruct 3D Gaussian representations for vehicles using sparse one-sided observations. To achieve metric-scale reconstruction, a visual foundation model is first utilized to serve as a visual prior for Gaussian initialization. The Gaussian attributes are then estimated by a learnable encoder-decoder module. A symmetry-aware cloning strategy is presented to complete the unobserved side directly in Gaussian space, which exploits the bilateral structure of vehicles as a geometric prior. Experiments on the public dataset demonstrate that the proposed method significantly outperforms existing approaches in both vehicle asset completeness and geometric accuracy.