Point cloud transformers made faster and easier to tune
Position Anchor Tuning: Towards Efficient Adaptation of Pre-Trained Point Cloud Transformers
Computer Vision and Pattern Recognition
Summary
Fine-tuning large pre-trained models for 3D point cloud data can be slow and expensive. The authors propose a new method called position anchor tuning that speeds up this process by reducing the amount of data these models have to process, using smart shortcuts that pick out key parts of the data. Their technique keeps the performance high while making the computations cheaper and requiring fewer adjustments. This could help make 3D data analysis on devices more efficient without needing a lot of computing power.
What this means in practice
- •For 3d vision engineers: Implement faster and lighter fine-tuning of pre-trained point cloud transformers for tasks like 3D object recognition or scene understanding.
- •For mobile device developers: Enable efficient deployment of 3D perception models on resource-constrained devices by reducing inference costs during model adaptation.
Authors
Zheng Liu, Xin Gao, Jinchao Zhu, Gao Huang
Abstract
Parameter-efficient fine-tuning (PEFT) has recently emerged as a pivotal research direction for adapting pre-trained point cloud transformers to diverse downstream tasks. Although existing methods achieve excellent fine-tuning performance with high parameter efficiency, they ignore inference efficiency. To tackle this problem, a novel PEFT method termed position anchor tuning (PAT) is proposed in this paper. As multi-head attention (MHA) and feed-forward network (FFN) are computation-heavy blocks in pre-trained transformers, PAT decreases their computational cost through token aggregation-expansion pairs. Each pair comprises a token aggregation module (TAM) and a token expansion module (TEM). For MHA and FFN blocks, TAMs extract representative tokens from their input tokens based on position anchors in 3D space. These extracted tokens, rather than the original input tokens, are processed by the blocks, thereby reducing the number of tokens involved in computation. Then, TEMs propagate the learned representations back to the original input tokens. Since TAMs are solely responsible for capturing task-specific representations, base-sharing low-rank adaptation (BSLoRA) is further introduced to enable them to learn such representations effectively with only a small number of trainable parameters. Extensive experiments on widely used benchmarks demonstrate that PAT performs comparably to state-of-the-art methods while incurring significantly lower computational overhead and fewer trainable parameters.