PlenoCI improves detecting changes in 3D visual scenes with fewer errors
PlenoCI: Plenoptic CharacterIstics for View Dependence Aware Change Classification
Computer Vision and Pattern Recognition
Summary
Detecting changes in complex 3D scenes is hard because small differences in visual data can look like changes even when the scene didn’t change. The authors introduce PlenoCI, a new way to capture visual details that focus on real structural changes and ignore simple texture differences. This method greatly reduces false alarms compared to previous techniques. It helps not only spot changes but also tell if they are because of geometry or appearance.
What this means in practice
- •For augmented reality developers: Improve detection of real environmental changes while ignoring textural noise to create more reliable AR scene updates.
- •For robotics teams: Accurately classify environmental changes as object movement or lighting variations to enhance robot perception and decision making.
Authors
Jason Lai, Chamuditha Jayanga Galappaththige, Niko Suenderhauf, Dimity Miller, Donald G. Dansereau
Abstract
Radiance field representations such as 3D Gaussian Splatting (3DGS) natively encode complex visual phenomena such as occlusions and view dependence, but they are inherently underconstrained. Independently optimized reconstructions converge to different primitive configurations, even in unchanged regions. We introduce Plenoptic CharacterIstics (PlenoCI), a novel feature built from the plenoptic field these representations approximate. PlenoCI directly captures rich visual behaviors while ignoring Lambertian textures. By deriving closed-form analytic plenoptic derivatives from a 3DGS representation, we efficiently detect these 5D structures. Our approach is robust to underconstrained representations by construction, reporting two orders of magnitude fewer false positives between independent reconstructions of unchanged scenes than concurrent work. We demonstrate PlenoCI's utility on change classification. First, we detect changes with an instance-aware 3DGS pipeline, achieving state-of-the-art results on CL-Splats with a 25.7% mIoU gain over the strongest competitor, while remaining competitive on the more challenging PASLCD benchmark. Leveraging PlenoCI, we classify changes as geometric or appearance-based with a balanced accuracy of 0.735, comparable to the best performing baseline. We believe plenoptic derivatives and PlenoCI open new directions for view dependence aware understanding in visually complex environments. Code and data are available at https://js0n-lai.github.io/plenoci.