Robotic pruning system learns to cut fruit trees using camera vision
Visuomotor Robotic Pruning in Planar Orchards Using Hybrid Reinforcement Learning
RoboticsArtificial Intelligence
Summary
Pruning fruit trees by hand takes a lot of work but is important for growing lots of fruit. This paper shows how a robot can learn to prune certain types of apple and cherry trees trained into flat shapes using just camera images, instead of complicated 3D scans. The robot is trained completely in a computer simulation and then tries pruning in real orchards without extra training, with success rates near half the time. This method guides the cutting tool through branches to the right spot and angle using a new way of combining learning from demonstrations and self-practice in simulation.
What this means in practice
- •For orchard managers: Automate pruning in high-density apple and cherry orchards using a vision-based robotic arm trained only in simulation and deployed without extra tuning.
- •For industrial robotics teams: Build robotic arms with wrist cameras that learn closed-loop control from hybrid reinforcement learning to navigate cluttered environments for precise cutting tasks.
Authors
Abhinav Jain, Cindy Grimm, Stefan Lee
Abstract
Dormant tree pruning is labor-intensive yet essential for maintaining modern high-productivity fruit orchards. In this work, we focus on pruning of modern planar tree training systems - V-Trellis apples and UFO cherries - where trunks and primary branches are trained into approximately planar walls. We introduce an end-to-end pipeline to learn a closed-loop visuomotor controller for robotic pruning. This controller is trained entirely using simulation and synthetically generated data and deployed in real orchards in a zero-shot manner. The pipeline comprises synthetic generation of planar orchard tree meshes, construction of a physics-based orchard simulator, automated collection of successful pruning trajectories via motion planning, and policy learning with a novel hybrid reinforcement-learning algorithm that combines offline demonstrations with online simulated rollouts. The controller uses optical-flow inputs from a wrist-mounted camera - avoiding the need for full 3D-reconstruction - and continuously guides the cutter through cluttered branch environments to a specified cutpoint with correct tool orientation. In exhaustive simulated task-space evaluations over 3,000 pruning points, the policy attains 49.9% success on V-Trellis apples and 46.0% on UFO cherries. We validate the learned controller across 38 physical trials - comprising 28 outdoor field trials in commercial and experimental orchards and 10 indoor laboratory tests - demonstrating zero-shot sim-to-real transfer. The learned policy also outperforms a classical RRT-Connect baseline on physical hardware in laboratory trials.