PQR3D improves multi-view 3D object detection with temporal refinement
PQR3D: Progressive Query Refinement over Reference-Conditioned Temporal Windows for Multi-View 3D Object Detection
Computer Vision and Pattern Recognition
Summary
Detecting 3D objects from multiple camera views over time is tricky because existing methods rely on remembering past frames in order. The authors propose PQR3D, which processes a group of frames at once and refines detection queries progressively without needing to remember everything continuously. This allows the system to analyze randomly sampled frames independently and still use past information effectively. Their approach improves accuracy on a popular self-driving car dataset.
What this means in practice
- •For autonomous vehicle developers: Use PQR3D to improve 3D object detection accuracy from multi-camera setups without requiring continuous frame order during inference.$Commercial implications: Enables advanced perception systems for self-driving cars that can process frames flexibly, improving safety and performance.
- •For surveillance system engineers: Apply PQR3D to detect and track 3D objects across multiple camera views without needing strict time sequence processing.
Authors
Hui Ye, Yudong Liu, Yiran Chen, Rajshekhar Sunderraman, Shihao Ji
Abstract
Temporal context is essential for camera-only multi-view 3D object detection. Existing streaming detectors maintain and propagate query states from one frame to the next, requiring sequence-aware training and chronological inference. We propose PQR3D, which performs progressive query refinement within referenceconditioned temporal windows. This design enables random frame sampling and independent inference without persistent query memory. Within each window, PQR3D progressively transfers motion-aligned high-confidence queries from earlier timestamps toward the target frame. We further introduce masked selfattention to regulate interactions among regular, propagated, and denoising queries while keeping denoising supervision isolated from detection queries. In addition, a stage-decoupled anchor embedding injects position before self-attention and size, orientation, and velocity afterward, reducing interference from temporally inconsistent attributes. With a ViT-L backbone, PQR3D sets a new state of the art on the nuScenes test set, achieving 71.6 NDS and 64.9 mAP. Source code is available at https://github.com/huiyegit/PQR3D