ElasticFit improves 3D object placement by fitting objects precisely
ElasticFit: Fit-Aware 3D Object Insertion via VLM Reasoning and Generative Adaptation
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Placing objects into 3D scenes is tricky because the new object must fit the space properly without breaking the scene’s logic. The authors developed ElasticFit, a system that uses language instructions and images of the scene to figure out where and how an object should be placed, including its size and orientation. ElasticFit then creates and adjusts the 3D object so it fits well without collisions or floating. This approach improves accuracy in placing objects compared to previous methods, especially when the object needs to be resized or reshaped to fit tight spots.
What this means in practice
- •For virtual environment designers: Insert customizable 3D objects into virtual scenes that fit realistically in complex spaces based on textual instructions.
- •For augmented reality developers: Place virtual objects in real-world scans with proper scaling and orientation to match physical constraints and support relationships.
Authors
Tzu-Hsin Hsieh, Ricardo Marroquim
Abstract
Inserting objects into existing 3D scenes requires more than selecting a plausible location: the inserted object must also fit local geometry while preserving semantic intent and physical plausibility. Although recent Vision-Language Models (VLMs) and generative models enable semantic reasoning and visual content creation, they offer limited 3D grounding and geometric control when an inserted object must fit into constrained local spaces. We introduce \textbf{ElasticFit}, a VLM-guided framework for fit-aware object insertion centered on a novel scene-grounded representation. Given a language instruction and rendered scene observations, ElasticFit infers structured fitting cues that specify where the object should be grounded, what volume it should occupy, how it should be oriented, and its adaptation mode (rigid placement, uniform scaling, or elastic fitting). These cues convert high-level VLM reasoning into explicit 3D constraints that condition object generation and guide downstream geometric fitting. ElasticFit then generates a scene-conditioned object prior, reconstructs it in 3D, and refines the mesh through mode-specific fitting while enforcing collision avoidance, contact consistency, and physical grounding. In fixed-asset baseline comparisons, ElasticFit improves spatial relation success from 50.8\% to 69.7\% and support success from 48.3\% to 91.7\% over the strongest baseline, while providing novel support for generative "make-it-fit" insertions in complex scenarios.