Spatial foundry provides rich 3d home scenes with linked visuals
Spatial Foundry: A Multimodal Dataset for Spatial Intelligence
Computer Vision and Pattern Recognition
Summary
Understanding how things in a room relate to each other is hard when you only have pictures or isolated notes. The authors created Spatial Foundry, a large collection of detailed 3D home scenes with furniture and many types of linked data like floor plans, photos, depth maps, and camera paths. This linked information makes it easier to study how rooms look, change, and are seen from different angles over time. The data also lets people edit scenes and explore how objects and cameras move together.
What this means in practice
- •For 3d graphics developers: Generate detailed 3D home scenes from floor plans using linked visual and geometric data for enhanced realism and control.
- •For robotics software teams: Model and predict camera and object movement in home environments with synchronized video and 3D data for improved navigation and perception.
Authors
Ethan Lin, Yang Song
Abstract
Spatial intelligence requires connecting visual observations to the geometry, objects, and persistent state of an environment. Images alone leave these relationships implicit, while disconnected annotations make it difficult to study appearance, camera motion, and scene changes under a common reference. We introduce Spatial Foundry, a collection of 5,280 furnished residential 3D scene configurations with editable assets and aligned visual, geometric, and temporal data. Its modalities include floor plans, perspective and panoramic RGB, metric depth, point clouds, meshes, instance-level 3D bounding boxes, and camera trajectories paired with video. Shared scene coordinates link observations across views and time; editable source scenes enable additional camera paths and controlled object changes. We describe the data representation and construction workflow, and discuss applications in floor-plan-conditioned 3D generation, geometric reconstruction, instance-level perception, camera-conditioned video generation, shared world modeling, and scene editing. Controlled camera and scene variations support spatial perception and world-model research. The public project page provides videos and an interactive colored point-cloud preview.