SiZeUp: Fast 3D Proxy from Aerial Images via Depth Ordinal Loss
2026-08-24 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors present SiZeUp, a fast way to create simple 3D models of buildings using aerial photos taken from an angle. They simplify the problem by representing each building with its footprint and a single height, then optimize this height using a special loss that compares the relative depth order of buildings from multiple views. This method avoids complicated point cloud data and unreliable absolute depth by focusing on relative depth, making the process much faster and stable. It achieves similar quality results to current methods but runs 23 to 52 times faster, useful for modeling large urban areas.
3D building reconstructionoblique aerial imageryheight-from-footprintordinal depth consistency lossmonocular depth estimationdifferentiable renderingmulti-view depth imagesproxy modelingdynamic view selectionscale ambiguity
Authors
Wenjun Zhou, Yunshan Li, Qiaoyu Zhu, Weidan Xiong, Hao Zhang, Daniel Cohen-Or, Hui Huang
Abstract
We present SiZeUp, a fast and scalable approach for constructing large-scale 3D urban proxy models directly from calibrated oblique aerial imagery. Our method adopts a height-from-footprint representation, reducing 3D building abstraction to a low-dimensional optimization problem in which building footprints are extruded by a single height parameter. To enable efficient and robust height estimation, we introduce an ordinal depth consistency loss that enforces agreement between the relative depth ordering of rendered proxies and depth priors predicted by a monocular depth model. This is realized through a differentiable renderer that maps parametric building proxies into multi-view depth images, allowing gradients to be propagated from depth supervision to building heights. Our ordinal formulation produces stable optimization in practice and avoids explicit feature matching or dense point cloud reconstruction. Rather than relying on metric depth, which can be unreliable under monocular scale ambiguity, our ordinal depth consistency loss operates on relative depths, providing a more reliable signal across views. Combined with an efficient dynamic view selection, our approach achieves a 23-52$\times$ speedup over state-of-the-art proxy reconstruction pipelines while maintaining comparable proxy-level coverage and volume consistency, making it well suited for large-scale urban modeling tasks.