AI summaryⓘ
The authors note that current 3D Gaussian Splatting methods can answer clear, direct questions about 3D scenes but struggle with understanding more complicated, hidden meanings or reasoning about objects and space. To tackle this, they created new tests called Causal-LERF and Causal-ScanNet that check if models can handle common sense and spatial reasoning in 3D. They found that existing methods don’t do well on these tests, so they developed a new system, CausalSplat, which combines vision-language models with 3D scene graphs to better separate what the scene shows from the logical thinking needed. Their experiments show CausalSplat works better on these reasoning tasks and still performs well on normal 3D segmentation tasks.
3D Gaussian SplattingOpen vocabulary scene understandingCommonsense reasoningSpatial reasoningAffordance reasoningCounterfactual reasoningVision-language models3D scene graphs3D segmentationCausalSplat
Authors
Jiayu Ding, Meilu Song, Yun Chen, Wei Gao, Ge Li
Abstract
While 3D Gaussian Splatting (3DGS) has advanced open vocabulary scene understanding, existing methods remain confined to explicit queries. They struggle to interpret implicit intents, complex spatial constraints, and commonsense reasoning required for practical embodied interactions. To address this gap, we introduce the task of reasoning 3D Gaussian segmentation and construct two benchmarks, Causal-LERF and Causal-ScanNet. These benchmarks systematically evaluate commonsense, spatial, affordance, and counterfactual reasoning. Evaluations reveal that current state of the art methods perform poorly on these reasoning challenges. Therefore, we propose CausalSplat, a framework that integrates vision-language models with 3D scene graphs to disentangle explicit structural perception from implicit logical inference. Extensive experiments demonstrate that CausalSplat achieves state of the art performance on our reasoning benchmarks while showing strong generalizability on standard referring and open vocabulary 3D segmentation tasks. Project Page: https://jiayuding031020.github.io/CausalSplat