Brain inspired recurrent model improves 3D point cloud completion
BRICE: Brain-inspired Recurrent Illusory Contour Encoder for Point Cloud Completion
Computer Vision and Pattern Recognition
Summary
Completing missing parts of 3D shapes is hard because existing methods mainly use straightforward processing and stack many attention steps, which can make models complicated and less understandable. The authors took inspiration from how the brain sees shapes by combining forward and backward signals to imagine missing contours. They created a new model that mimics these brain loops to better guess missing parts of 3D scans. This approach can be added to existing 3D shape completion systems and improves their results on several datasets.
What this means in practice
- •For 3d vision engineers: Enhance existing 3D point cloud completion pipelines by integrating a brain-inspired recurrent module to better infer missing geometry in partial scans.
- •For robotics system builders: Improve object shape estimation from incomplete sensor data using a feedback-based neural design for more accurate environment interaction.
Authors
Yunlei Wang, Dawei Li, Xue-song Tang
Abstract
Mainstream point cloud completion methods are unanimously based on deep learning architectures. On one hand, these methods tend to repetitively stacking attention modules to enhance network performance, which drifts away from the neat design of unified and interpret-able networks; on the other hand, nearly all existing architectures rely solely on feed-forward connections to propagate information, neglecting feedback from deeper layers. The recurrent neural architecture, formed by feedforward-feedback loops, have shown its ability to generate Illusory Contours (ICs) in the V1 brain region of mammals; and ICs are essentially a completion mechanism of the brain for visual information. Motivated by this, we design a Brain-inspired Recurrent Illusory Contour Encoder (BRICE) to "imagine" what is missing in the incomplete 3D point clouds. Building upon the foundation model of Predictive Coding (PC), the plug-and-play nature enables BRICE to be seamlessly integrated into mainstream point cloud completion baselines. Experimental results show that BRICE can enhance various completion baselines on different 3D datasets such as MVP2K, PCN, and PlantPCom. Our code: Https://github.com/wangyunlei-wyl/BRICE.