Lightweight instance method improves 4D occupancy forecasting accuracy
LighTROcc: Lightweight 4D Occupancy Forecasting via Instance-Centric 3D Gaussians
Computer Vision and Pattern Recognition
Summary
Forecasting where objects will be in 3D over time is important for self-driving cars to understand their surroundings. Existing methods use dense 3D grids that get very expensive as they try to be more detailed and predict further into the future. The authors introduce LighTROcc, a lighter approach that tracks individual movable objects using a few learned points and models their shape as 3D Gaussian blobs. This method predicts current and future object locations efficiently and keeps the identity of each object over time. Tests show it is more accurate at forecasting object-level positions while still providing good overall 3D occupancy quality.
What this means in practice
- •For autonomous vehicle engineers: Integrate compact instance-centric occupancy forecasts from cameras to improve future object positioning and tracking in self-driving cars.
- •For robot navigation teams: Use efficient 3D occupancy predictions to enhance navigation and path planning around dynamic objects in real time.
Authors
Hwanhee Jung, SeungHyeon Kim, Inkyu Koo, Qixing Huang, Sang Ho Yoon, Sangpil Kim
Abstract
Forecasting future 3D occupancy from surround-view cameras is essential for autonomous driving, yet existing approaches rely on dense voxel or bird's-eye-view representations whose cost grows rapidly with spatial resolution and prediction horizon. Because these representations do not explicitly maintain object identities, they also struggle to preserve instance consistency over time. We present LighTROcc, a lightweight instance-centric framework that represents movable objects with a compact set of learned queries and predicts present and future occupancy in a single forward pass. LighTROcc localizes each query through attention-guided forward lifting, combining image-space cross-attention, query-specific depth, and camera geometry to estimate its 3D center. Each instance is modeled as a mixture of anisotropic 3D Gaussians and propagated across future steps using predicted displacements, producing continuous, temporally consistent occupancy forecasts. Experiments on nuScenes and supplemented nuScenes-Occupancy show that LighTROcc outperforms the evaluated dense and instance-wise baselines in instance-level forecasting accuracy while maintaining strong voxel-level occupancy quality. Across different model configurations, LighTROcc achieves a favorable balance between forecasting accuracy and computational efficiency, demonstrating the potential of compact instance-centric modeling for camera-based 4D occupancy forecasting.