PanoVine: Whole-Body Visuomotor Control for Soft Growing Vine Robot
2026-06-22 • Robotics
Robotics
AI summaryⓘ
The authors developed a soft robot called a vine robot that can grow and move through tight and tricky spaces. These robots are hard to control because they bend and interact with their tethers unpredictably. To solve this, the authors used 19 cameras along the robot's body to see what it’s doing and where it is. They trained a computer program using these camera views to help the robot move on its own in tricky places, like climbing slopes or squeezing through small gaps. Their tests showed this method helps the robot navigate and grab things more reliably.
soft roboticsvine robotsvisuomotor controlclosed-loop controldistributed sensingautonomous navigationend-to-end learningrobot manipulationhysteresisrobot perception
Authors
Yimeng Qin, Xiaomeng Xu, William Heap, Aditi Oak, Shuran Song, Allison Okamura
Abstract
Vine robots, a class of soft, growing robots, are suitable for navigating complex and confined environments due to their compliant bodies and self-supporting growth mechanism. However, hysteresis, tether interactions, and deformations make them difficult to predict and model, which in turn limits the effectiveness of conventional planning and control approaches. In this work, we present a data-driven, vision-based control framework for the first autonomous vine robot system. Our system integrates 19 cameras distributed along the robot's body to provide comprehensive feedback of both the robot state and the surrounding environment. Using this rich whole-body vision feedback, we train an end-to-end visuomotor policy from demonstrations for closed-loop autonomous control in complex environments. The policy efficiently aggregates information from distributed sensing while maintaining robustness to inaccurate robot states and actuation. Experimental results demonstrate that the learned policy enables robust navigation and manipulation in challenging scenarios, including steering through branched structures, climbing up slopes, traversing unsupported terrain, reaching objects precisely, and maneuvering through confined spaces and obstacles. Project website https://panovine-bot.github.io