Robots learn to adaptively build block structures with precision
Learning to build covering structures with continuous adjustments
RoboticsMachine Learning
Summary
Building complex structures with robots can be tricky because real-world materials and measurements aren’t perfect. The authors developed a new method that lets robots learn how to place blocks step-by-step, adjusting their plans as they go rather than following a fixed blueprint. They use a special computer program that understands connections in the blocks and makes smart decisions about which block to place and exactly where to put it. This approach works better than previous methods, learns efficiently, and can be applied from computer simulations to real robots building actual blocks.
robotic constructionreinforcement learninggraph neural networksparameterized actionssoft actor-critic (SAC)simulation to real transferadaptive planningspanning archcontinuous adjustmentsample efficiency
Authors
Gabriel Vallat, Maryam Kamgarpour, Stefana Parascho
Abstract
Robotic construction offers the potential to use materials more efficiently and create complex geometries, but current methods rely on rigid, high-precision plans that cannot accommodate the tolerances, inaccuracies, and unexpected changes inherent in physical fabrication. In this work, we introduce a reinforcement learning approach that forgoes predefined plans entirely, instead generating construction sequences adaptively as the structure is built. Our method operates on graph-structured state representations and a mixed (parameterized) action space, requiring both discrete block selection and continuous placement parameters. Because the stability simulation of a structure is computationally heavy, we develop an efficient exploration strategy by incorporating unilateral edges into graph neural networks, extending soft actor-critic (SAC) to this hybrid setting. We evaluate our algorithm, HSAC, against the prior method hybrid-PPO (HPPO), demonstrating significantly higher asymptotic performance and good sample efficiency. We also demonstrate HSAC's robustness to hyperparameter choices and its exploration capability, handling up to 10 discrete actions without performance degradation. Finally, we validate our approach on a physical two-robot setup, successfully building a spanning arch with 3D-printed blocks in closed-loop execution, confirming that policies trained in simulation transfer to real hardware.