Multimodal AI learns 3D spatial reasoning from new multi-view dataset
MV-STRIDE: Enabling MLLMs to Master Multi-View Spatial Reasoning via Hierarchical Capability Modeling
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Understanding how objects relate to each other in three dimensions from different viewpoints is hard for current AI systems. The authors created a new dataset called MV-STRIDE that teaches AI models to think step-by-step about spatial relationships using multiple views and 3D scenes. This dataset mimics how humans understand space by building up from basic perception to more complex reasoning. Using this approach, AI models improved their ability to solve spatial problems that need understanding from many angles. Their method shows better results on tests that involve reasoning about objects in 3D space.
Multimodal Large Language ModelsSpatial reasoningMulti-view datasets3D scene understandingHierarchical modelingChain-of-thought supervisionVision-language tasksCross-view constraintsBenchmark evaluation
Authors
Jin Xu, Xiaojian Huang, Zhuodong Luo, Zhihong Zhang, Xin Liu, Jiansheng Wei, Xinzhi Wang, Jie Zhao, Xuejin Chen
Abstract
Despite the rapid progress of Multimodal Large Language Models (MLLMs) in 2D vision-language tasks, robust multi-view spatial reasoning remains a fundamental bottleneck due to the lack of structured 3D cognitive pathways in existing datasets. To address this, we introduce MV-STRIDE, a Multi-View hierarchical SpaTial Reasoning dataset with Interdependent and DEcomposed capabilitiEs. Moving beyond flat data structures, MV-STRIDE explicitly models the dependency relationships between foundational perception, scene understanding, and complex contextual reasoning, providing a coherent learning pathway aligned with human spatial cognition. We develop a systematic QA generation pipeline leveraging diverse 3D scene sources that enforces cross-view dependency constraints to prevent single-view solvability, generating multi-level spatial reasoning tasks supported by cognitively grounded chain-of-thought supervision for complex inference. Extensive evaluations demonstrate that our multi-stage training framework based on our hierarchical dataset achieves state-of-the-art performance across multiple spatial reasoning benchmarks, notably the multi-view oriented MMSI-Bench. Our approach enables MLLMs to maintain robust, 3D-consistent spatial reasoning across diverse viewpoints. The code and dataset are available at https://co1dspring.github.io/MV-STRIDE/.