System creates relightable 3D garments from marked sketches
From Sketch to Scene: Relightable Garment Reconstruction from Marked Sketches
GraphicsComputer Vision and Pattern RecognitionHuman-Computer Interaction
Summary
It can be hard for current computer programs to turn sketches of clothing into 3D models with realistic lighting that can be changed later. The authors propose a new method where a designer draws stars and moons on a simple sketch to show where highlights and shadows should appear. Then, the system builds a 3D model of the garment with a light source that matches those marks and shapes the surface so shadows fall correctly, making the model easy to relight and edit. This helps bring rough fashion sketches to life with more control over lighting effects.
What this means in practice
- •For fashion designers: Create editable 3D garment models from simple sketches with user-indicated lighting for faster prototype visualization.$Commercial implications: Enables design tools that generate relightable 3D clothes directly from designers' annotated sketches for quicker creative iterations.
- •For game developers: Generate relightable clothing assets from annotated sketches to streamline the creation of detailed character outfits.
Authors
Ahmet Neçirvan Doğan, Eren Acar Başaran, Metin Sezgin, Zahra Montazeri
Abstract
Sketching is one of the most natural ways for people to express visual ideas. Unlike the text prompts that most generative systems rely on today, a sketch captures spatial structure and intent directly, including where a designer wants highlights and shadows to appear. Yet current sketch-to-3D pipelines have no input for that intent: they bake the drawn shading into the texture as dark paint, and the light cannot be moved. We propose a system that transforms a rough garment sketch, marked with sparse annotations (a star for a highlight and a moon for a shadow), and a short text prompt describing the texture, into a textured 3D mesh with an explicit directional light, as a scene that can be relit and edited after generation. The two symbols are the most frequent proposals in an elicitation study with art students. From the marked sketch the system predicts the scene's directional light and, separately, a displacement field that bends the surface so that the moon-marked regions fall into shadow under that light, rather than being painted dark in the texture.