AQ3D: Adaptive Query Transformer for 3D Instance Segmentation
2026-08-31 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address the problem that existing 3D instance segmentation models use a fixed number of guesses (queries) and position information that don't adjust well to different indoor scenes, which vary a lot in size and number of objects. They propose AQ3D, a method that creates queries based on the scene's size, using superpoints as building blocks, and encodes positions in a flexible way using 3D rotary positional encoding. They also improve the model's decoder with better pooling, mask refinement, and background handling to improve accuracy. Their approach performs better than previous methods on several 3D indoor scene datasets without extra data tricks.
3D instance segmentationtransformer decoderqueriessuperpointspositional encoding3D rotary positional encodingmask refinementcosine classifierScanNet datasets
Authors
Keno Moenck, Thorsten Schüppstuhl
Abstract
Transformer-based decoders for 3D instance segmentation typically commit to a fixed number of queries and positional modeling calibrated on the training distribution rather than on the scene at hand. Indoor scans vary widely in spatial extent and object count, so a fixed query set over-initializes small scenes and under-initializes large ones, while learned absolute and relative encodings are bound to the training scenes' extents and can saturate. We present AQ3D, which is designed to handle scenes of various sizes during training and inference. Queries are instantiated at a fixed ratio of the scene's superpoints, forming an overcomplete set whose background rejection is entirely left to the decoder. Positional information is encoded using 3D RoPE over quantized metric coordinates, replacing learned bounded lookup tables of prior decoders. Further, we improve the decoder itself by using attribution-based superpoint pooling, a mask refinement branch, and a cosine classifier for background rejection. Experiments show our method sets a new state-of-the-art on validation and hidden test splits across the datasets ScanNetV2, ScanNet200, and ScanNet++V2 among decoder methods trained without additional data augmentation. Code is available at \href{https://github.com/kenomo/aq3d}{github.com/kenomo/aq3d}.