MindView benchmark tests AI on generating consistent 3D views from partial data
MindView: Benchmarking Generative Multi-View Spatial Intelligence
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Knowing how things look from different angles is hard for computers. The authors created MindView, a set of tasks that check how well AI can imagine or generate images of indoor scenes from new viewpoints using only partial information. They tested several AI models and found that most struggle with this problem but improved one significantly by training it with their new data. Their work also shows that learning to generate these new views helps with general spatial understanding.
What this means in practice
- •For computer vision teams: Evaluate and improve models that generate consistent images from multiple viewpoints in indoor environments using partial observations.
- •For robotics engineers: Use the benchmark and methods to enhance robots' ability to visualize unseen views of indoor spaces, improving navigation and spatial reasoning.
Authors
Yanbo Wang, Siqi Luo, Siyu Zhang, Chenming Zhu, Wenzhao Zheng, Qian Yu, Juncheng Yan, Bin Fu, Tai Wang, Yu Qiao, Yihao Liu, Jiwen Lu
Abstract
Spatial intelligence has recently been assessed through generative tasks such as spatially grounded image editing, yet language-guided generation of spatially consistent views from partial multi-view observations remains underexplored. We introduce MindView, a benchmark for generative multi-view spatial intelligence with 1,500 samples from 513 scenes across three indoor datasets. Its seven tasks form two families: geometry-dominant tasks specify changes in camera position and orientation, while semantics-dominant tasks require inferring target viewpoints by grounding referred objects and interpreting spatial relations and reference-frame cues. Evaluation of eleven generators reveals substantial room for improvement on MindView. A paired context analysis further shows that all eleven generators obtain lower overall scores under the expanded context setting. These findings motivate us to investigate whether task-specific supervision can improve target-view generation and whether models can use additional multi-view evidence effectively. We therefore fine-tune BAGEL on the MindView training dataset, improving its overall score by 24.3 points and yielding gains across all seven MindView tasks. Beyond these generation gains, the fine-tuned model also yields higher scores on three spatial-understanding benchmarks without training on their annotations, suggesting that learning to generate spatially grounded views may benefit spatial understanding. To improve evidence utilization under expanded context, we propose MindCue, a training-free spatial evidence selector guided by a target-view hypothesis. MindCue reverses the drop in overall score under expanded context and outperforms the standard-context baseline. Overall, MindView provides a concrete testbed for generative multi-view spatial intelligence.