OrientSAM: Mitigating Camera-Centric Shortcut in Multimodal Spatial Reasoning via Orientation-Aware Spatial Alignment
2026-07-20 • Artificial Intelligence
Artificial IntelligenceComputer Vision and Pattern RecognitionMultimedia
AI summaryⓘ
The authors found that current multimodal language models often make mistakes when trying to understand spatial relationships from perspectives other than the camera’s. This happens because the models focus too much on the camera viewpoint instead of the object’s own orientation. To fix this, the authors created OrientSAM, a method that adds clear information about object orientation into the model’s understanding and trains it step-by-step to improve perspective reasoning. Their tests showed OrientSAM works better, especially in tasks where understanding different viewpoints is important.
multimodal large language modelsspatial reasoningperspective transformationcamera-centric biasobject orientationorientation encodingcurriculum learningallocentric reasoningspatial alignment
Authors
Wenxiao Fan, Hang Yin, Kan Li
Abstract
Multimodal large language models (MLLMs) still struggle with spatial reasoning that requires perspective transformation. In particular, they often rely on camera-centric cues rather than reasoning from the reference object's viewpoint, leading to systematic errors in non-camera reference settings. In this paper, we first analyze this failure mode and show that object orientation is a key factor underlying such camera-centric shortcut behavior. To address this issue, we propose OrientSAM, an orientation-aware spatial alignment framework for multimodal models. OrientSAM injects explicit orientation information into multimodal representations through orientation-aware tokens and Fourier-based angle encoding, and further adopts a curriculum learning strategy to progressively improve perspective-aware reasoning. In addition, we build a spatial data construction pipeline to generate orientation-aware spatial supervision from large-scale images. Experiments on Spatial-MM, ViewSpatial, and 3DSRBench show that OrientSAM consistently outperforms strong baselines, especially on non-camera-view, person-centric, and orientation-sensitive tasks. The results further demonstrate that explicit orientation modeling is important for mitigating camera-centric shortcut behavior and enabling more robust allocentric spatial reasoning in multimodal models.