Wide view images from multiple car cameras without per scene tuning
Wide View Synthesis from Overlapping Perspective Cameras in Autonomous Driving
Computer Vision and Pattern Recognition
Summary
Autonomous vehicles use many cameras to see around them, but combining their views into one big picture is hard because the cameras see from different places and angles. The authors created a new method called Wide-View Gaussian Splatting (WV-GS) that can quickly make a consistent wide image from several cameras without needing extra adjustments for each scene. They also introduced new techniques to reduce distortion at image edges and fix brightness differences between cameras. To test their method, they made a synthetic dataset and used real driving data, showing their approach works well.
What this means in practice
- •For autonomous vehicle engineers: Generate unified wide-angle views from car cameras in real time without scene-specific adjustments to improve vehicle perception.
- •For robotics perception teams: Produce geometrically consistent wide views from separated cameras for robots navigating complex environments.
Authors
Kaixuan Zhang, Zhipeng Xiong, Minxian Li, Mingwu Ren, Xiatian Zhu
Abstract
Multi-perspective-camera systems are widely adopted since they provide broad visual coverage for autonomous driving, motivating the synthesis of a unified wide view from spatially separated observations. In this work, we formulate wide view synthesis (WVS), which aims to synthesize a geometrically consistent wide view at a specified virtual viewpoint from multiple perspective cameras. Unlike conventional image stitching, WVS requires reasoning about 3D geometry and visibility to handle depth-dependent parallax and occlusions arising from spatially separated camera centers. While vanilla 3D Gaussian Splatting (3DGS) enables high-fidelity view synthesis, its reliance on per-scene optimization limits its applicability to driving scenes. Furthermore, WVS introduces additional challenges, *e.g.*, peripheral stretching under perspective projection and photometric inconsistencies across cameras. To address these challenges, we propose **Wide-View Gaussian Splatting (WV-GS)**, a feed-forward Gaussian-splatting framework that synthesizes wide views without per-scene optimization. Specifically, we introduce Piecewise Perspective Rendering (PPR) to mitigate peripheral stretching through local perspective projections sharing a common camera center, and Spatial Affine Photometric Modeling (SAPM) to account for camera-dependent photometric variations during training. We further introduce **WideDrive**, a synthetic benchmark providing dense ground-truth wide views and adapt a LiDAR-projected sparse-reference protocol to real-world datasets. Extensive experiments on multiple datasets demonstrate the effectiveness of WV-GS for WVS. Dataset and code will be publicly released.