Robot-captured cloth images improve neural material models accuracy
RoboCloth: A Large-Scale Real Cloth Material Dataset for Neural Reflectance Reconstruction
Computer Vision and Pattern RecognitionGraphics
Summary
Capturing how cloth looks under different lights and angles is tricky but important for making realistic digital clothes. The authors created RoboCloth, a large set of photos of real cloth materials taken by a robot that carefully controls the camera and lights. They used this special data to train a neural network that better understands cloth appearance than models trained on older datasets. This means better digital rendering of fabrics is possible, even on new cameras and lights. They share the dataset and code openly so others can build on their work.
What this means in practice
- •For computer graphics developers: Train cloth rendering models that better mimic real fabrics under varied lighting using the RoboCloth dataset and calibration files.
- •For robotic vision engineers: Use robotic camera and lighting control techniques demonstrated here to acquire high-quality material appearance datasets.
Authors
Zhen Li, Haoran Qin, Francesco Di Sario, Daniel Rebain, Andrea Tagliasacchi
Abstract
We present RoboCloth, a dataset of 500 real cloth materials captured under flexible, calibrated camera and lighting configurations, where both are robotically controlled during acquisition. Each material is imaged in approximately 580 high-dynamic range frames spanning a wide view-light hemisphere; camera poses are refined via structure-from-motion and aligned to the robot frame, yielding per-material sparse surface geometry and structured per-point reflectance observations ready for neural material training. We demonstrate the value of this data through a two-stage neural reconstruction pipeline: Stage 1 trains a shared MLP decoder on our corpus to learn cloth material priors; Stage 2 then freezes the decoder and fits a dense 2048x2048 latent texture to each target material. Evaluating on held-out materials from our own dataset and on the independent UBO2014 benchmark, we show that a decoder trained on our captures outperforms decoders trained on existing measured reflectance datasets and analytic PBR fitting, transferring to unseen acquisition systems without modification. The dataset, calibration files, observation tensors, and train/test splits are released under CC BY 4.0 on the Hugging Face Hub.