InterOCF: Spatio-Temporal 2D-3D Interaction for Camera-Only 4D Occupancy Forecasting
2026-07-27 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors present InterOCF, a new method that helps self-driving cars predict what the 3D environment around them will look like in the near future using only camera images. Their method improves how the system understands changes over time by combining 3D voxel data with 2D image segmentation information. They created three parts: one to handle 3D changes, one for 2D temporal image analysis, and one to connect these two views. Their tests show InterOCF works better than previous methods on several standard datasets.
4D occupancy forecasting3D voxel representationmulti-view imagesspatio-temporal modelingsemantic segmentationautonomous vehiclescamera-only perceptiontemporal dynamicsnuScenes datasetLyft-Level5 dataset
Authors
Qi Zhang, Xinquan Yu, Kaiyi Zhang, Hui Huang
Abstract
Camera-only 4D occupancy forecasting enables autonomous vehicles to predict future 3D semantic scenes solely from historical multi-view images, which is critical for driving safety. Even though current methods have achieved good performance, the strong spatial-temporal modeling between the input multi-view frames is still underexplored, which limits the performance of those methods in future 4D forecasting. To address this gap, we introduce a novel framework, InterOCF, for 4D occupancy forecasting that jointly models temporal dynamics in both 3D voxel-based representations and multi-view segmentation sequences, while explicitly incorporating feature interaction between the 2D and 3D branches. Our framework incorporates three core components: 1) A 3D Spatio-Temporal (3DST) module that learns volumetric dynamics from historical voxel states to predict future voxel states; 2) A 2D Spatio-Temporal (2DST) module employing an auxiliary multi-view temporal segmentation forecasting task to enhance temporal semantic dynamics; 3) A Spatio-Temporal Interaction Modeling (STIM) module that enables feature interaction between 2D and 3D representations. Experiments on the nuScenes, Lyft-Level5, and nuScenes-Occupancy datasets show that InterOCF consistently outperforms existing baseline approaches.