Efficient tuning method improves 3D scene understanding in point clouds
Partition-Invariant Tuning for 3D Scene Understanding
Computer Vision and Pattern Recognition
Summary
Understanding 3D scenes from point clouds is hard because scenes can be big and complex. The usual way to adjust models for new scenes takes a lot of computer power and memory. The authors designed a new method called PointPiT that makes this adjustment smarter and more efficient by focusing on stable parts of the scene and ignoring confusing differences caused by how the data is split up. This lets models learn well from big scenes using very few changes, matching or beating more costly methods.
What this means in practice
- •For augmented reality developers: Enhance scene-level 3D understanding models to run efficiently on devices with limited memory and computing power.
- •For autonomous vehicle engineers: Improve 3D environment perception models by tuning them to varied large-scale scenes without heavy computational overhead.
Authors
Hongqiang Lin, Tianle Wang, Shuiwang Li, Dongxu Zhang, Yiding Sun, Zihao Guo, Dongfu Yin
Abstract
Scene-level point cloud understanding remains challenging due to diverse geometries and spatial layouts. While pre-trained 3D point cloud foundation models (PFMs) offer strong transferability, full fine-tuning (FFT) incurs substantial computational and storage costs. Parameter-efficient fine-tuning (PEFT) provides a promising alternative, but existing PEFT methods largely focus on object-level point clouds and overlook serialization-induced partition variations in large-scale scenes. To address this issue, we propose PointPiT, a partition-invariant tuning framework for scene-level point clouds. Specifically, a Scene-aware Structural Adapter (SSA) integrates local geometric patterns with global scene context to mitigate partition-induced representation shifts. Moreover, Gradient Subspace Optimization (GSO) selects informative and partition-stable update directions, suppressing partition-dependent variations during optimization. Extensive experiments across multiple scene-level benchmarks demonstrate that PointPiT achieves competitive or even superior performance to full fine-tuning with less than 1% of backbone's parameters, while achieving consistent state-of-the-art performance among representative PEFT methods.