SpatialBlock improves 3D understanding in vision language models
SpatialBlock: Enhancing Spatial Intelligence in LVLMs via Synthetic Block-Stacking Problem
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Vision-language models struggle to understand 3D shapes and layouts from flat images. To help with this, the researchers created a set of 15,000 computer-generated block-stacking problems that teach models about 3D shapes from different views and colors. Models trained on this dataset got better at solving tricky spatial puzzles and even performed well on real-world tasks despite the data being synthetic. This method is faster and cleaner than previous ways that relied on expensive and noisy real-world labels. The work shows that teaching models through structured, simple tasks can build better spatial skills.
Vision-language modelsSpatial intelligence3D reconstructionSynthetic datasetBlock stacking3D to 2D projectionViewpoint transformationAnchor-based reasoningVisual reasoningSpatial question answering
Authors
Soohyun Ryu, Sohee Kim, Eunho Yang
Abstract
Large Vision-Language Models (LVLMs) have achieved strong performance on diverse visual tasks, yet their ability to reconstruct and reason about the 3D structure of the scene depicted in 2D images -- referred to as spatial intelligence -- remains limited. Existing approaches attempt to address this gap by using real-scene spatial question answering datasets that require dense geometric annotations. However, constructing such labels is costly, time-consuming, and often noisy due to reliance on external perception modules. In this work, we propose a novel paradigm inspired by human cognitive development: learning foundational spatial skills through structured block-manipulation tasks. We introduce SpatialBlock-15k, a synthetic dataset of 15,000 block-stacking problems covering 3D-to-2D projection, viewpoint transformation, and structural combination. The dataset further incorporates controlled color modulation as visual cues to encourage anchor-based reasoning in visually complex conditions. Experiments demonstrate that LVLMs trained on our dataset through either direct answering or reasoning-based prediction significantly outperform baselines and generalize to real-world spatial tasks, despite the dataset's synthetic and compact nature. Code and data are available at https://github.com/rsoohyun/SpatialBlock.