A Cookbook of 3D Vision: Data, Learning Paradigms, and Application
2026-06-02 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors provide a clear overview of how 3D vision works by organizing different 3D data types like point clouds, meshes, voxels, and 3D Gaussians into one framework. They also explain how datasets, benchmarks, and learning methods influence progress in 3D tasks like reconstruction and video modeling. Their work helps connect the dots between various representations, learning techniques, and applications, showing current trends in making 3D models both efficient and accurate.
3D visionpoint cloudsmeshesvoxels3D Gaussiansdatasetsbenchmarkslearning paradigmsreconstructionimplicit neural representations
Authors
Hongyang Du, Zongxia Li, Dawei Liu, Runhao Li, Haoyuan Song, Qingyu Zhang, Yubo Wang, Jingcheng Ni, Shihang Gui, Congchao Dong, Tao Hu
Abstract
3D vision has rapidly evolved, driven by increasingly diverse data representations, learning paradigms, and modeling strategies. Yet the field remains fragmented across representations and benchmarks, making it difficult to develop unified perspectives on efficiency, fidelity, and scalability. This work provides a data-centric taxonomy of 3D vision that connects geometric representations, datasets, learning frameworks, and applications within a single conceptual map. We begin by analysing the principal structural representations of 3D data--point clouds, meshes, voxels, and 3D Gaussians--along with their acquisition pipelines. We then examine how dataset design, benchmark construction, and supervision regimes shape recent advances, spanning 2D-supervised 3D learning, implicit neural representations, and 4D world modeling. Through this integrative lens, we clarify the relationships among representations, learning paradigms, and downstream tasks in reconstruction, generation, and video modeling, offering a consolidated view of emerging trends toward balancing efficiency and fidelity and toward multimodal geometric grounding.