Summary
Inserting steel bars called rebar is a very repetitive and physically tough construction job that requires precise fitting. The authors created a simulated training system that teaches a robot to insert a variety of rebar shapes by using computer vision and rehearsal in simulation. Their approach successfully transfers the learned skill to a real robot without extra real-world training, achieving over 90% success in tests. Training the robot on diverse shapes and visual variations helped it handle new rebar designs better and adapt quickly to changes. This work aims to reduce manual effort and speed up construction tasks involving rebar installation.
What this means in practice
- •For construction robotics teams: Automate installation of diverse rebar designs on site by deploying a vision-based robot trained entirely in simulation.
- •For robotics software developers: Use simulation-trained visual policies and domain randomization to handle objects with geometric variations in contact-rich tasks.
Authors
Tao Sun, Beining Han, Patrick Yin, Rui Xu, Harry He, Abhishek Gupta, Szymon Rusinkiewicz, Yi Shao
Abstract
Rebar insertion is among the most repetitive and physically demanding tasks on construction sites, and a contact-rich problem at 1.4 mm clearance. The parts, however, vary at two levels: a nominal design per structural member, and fabrication tolerance around each nominal design. Real-world data therefore has to be re-collected as designs and batches change. We present RebarSim, a visual sim-to-real system trained entirely in simulation. A privileged state-based teacher is trained with reinforcement learning over procedurally generated rebar geometries, then distilled into a multi-view student that maps raw RGB and proprioception directly to actions under extensive domain randomization. The student transfers to the real world zero-shot, seating rebars taken from a real factory production run in 91.3% of real-robot rollouts. Underlying that result, geometry diversity and pretraining both bring benefits. Training across a diverse set of nominal designs rather than one lifts the zero-shot success of both the teacher and the student on unseen designs, and the student policy outperforms a single-design specialist on that specialist's own design. A pretrained student then adapts to a new design with 4--6x fewer distillation samples than one trained from scratch. Visual sim-to-real transfer depends on appearance randomization and the DAgger mixture: removing either one sharply lowers success. Videos, code, and task assets are available at https://rebarsim.github.io.