Feature diverse upsampling improves 3D scene occupancy prediction accuracy
FM-Occ: Feature-Diverse Upsampling and Memory-Aware Decoding for 3D Occupancy Prediction
Computer Vision and Pattern Recognition
Summary
Predicting which parts of a 3D space are occupied and by what type of object is challenging due to how neural networks upsample features and handle internal memory. The authors found that common upsampling methods make features too similar and lose good semantic guesses through decoder layers. They created two improvements: one reshuffles features more diversely to keep differences clear, and another keeps important past info in memory to avoid losing good predictions. These changes led to better accuracy on a standard 3D semantic dataset with little extra memory cost.
What this means in practice
- •For autonomous vehicle engineers: Improve 3D environment understanding modules by integrating feature-diverse upsampling and memory-aware decoding to reduce prediction errors in sparse voxel representations.
- •For robotics perception teams: Enhance robot navigation systems by using entropy-regulated memory mechanisms to maintain semantic information across processing layers for better scene interpretation.
Tested on one dataset.
Authors
Yujia Zhai, Guoming Huang, Zhuanghuai Wang, Bangquan Xie
Abstract
3D semantic occupancy prediction estimates voxel-wise occupancy and semantic labels throughout a scene. Our experiments reveal that conventional sparse transposed-convolution upsampling yields poorly differentiated voxel responses and that successive decoder layers discard previously correct semantic predictions. To address these issues, we present Feature-Diverse Upsampling and Memory-Aware Decoding for 3D Occupancy Prediction (FM-Occ), comprising two components. First, 3D Sparse Subvoxel Shuffle (3S) mitigates feature homogenization by constructing candidate child features in the channel dimension, jointly transforming and normalizing them, and then rearranging them onto a finer sparse grid with explicit feature-coordinate correspondence. Second, Entropy-Regulated Layer-wise Memory (ER-LM) retains semantic information across decoder layers by maintaining a layer-wise memory for each query through gated recurrent updates and using prediction entropy to regulate the historical information propagated to the next layer. During training, a class-weighted entropy loss Lcwe provides label-guided regularization for matched semantic queries. On SemanticKITTI, 3S increases the spatial coefficient of variation of normalized voxel-wise root-mean-square (RMS) responses from 0.41% to 4.03%, while ER-LM reduces semantic degradation and improves error correction across decoder layers. The combined FM-Occ improves mIoU from 12.16% to 12.73% with negligible memory overhead. ER-LM further improves mIoU in the BEV-based DISC framework from 16.78% to 17.17%, supporting its applicability across 3D voxel-based and BEV-based architectures.