Hydraulic excavator learns fast precise control in 40 minutes
Precision at Speed: Sample-Efficient Online Model-Based Reinforcement Learning for Hydraulic Excavator Control
RoboticsMachine Learning
Summary
Controlling big machines like excavators precisely and quickly is hard because they move in complex ways. The authors created a system that learns how to control a hydraulic excavator directly on the machine without needing prior training or demonstrations. Their method learns faster than existing approaches, reaching very accurate movement along paths within about 40 minutes of real machine time. This can help machines work both fast and precisely with less setup.
What this means in practice
- •For construction equipment operators: Enable excavators to quickly learn precise digging paths on real machines with minimal downtime using onboard learning.$Commercial implications: This paper enables selling self-tuning control software for hydraulic excavators that reduces training time and improves operational precision in construction.
- •For industrial robotics teams: Implement fast and accurate online learning controllers for robots with complex actuation dynamics without extensive simulation training.
Authors
Claudio Canales, Fang Nan, Marco Hutter, Javier Ruiz-del-Solar
Abstract
Precise, high-speed control remains challenging for robots with complex actuation dynamics. Learning directly on hardware is further constrained by the cost of real-world interaction. We present an online model-based reinforcement learning framework that learns a probabilistic dynamics ensemble model from scratch for sampling-based model predictive control. A precision-gated contouring objective conditions the progress reward on path accuracy, prioritizing precision over speed. In a data-driven excavator simulator, the framework achieves higher sample efficiency than the evaluated model-based reinforcement learning baselines. We validate the framework by learning directly on an 11.5-ton Menzi Muck M445 hydraulic excavator, without demonstrations or simulation pretraining. After 20 minutes of interaction, the controller reaches tracking accuracy comparable to prior learned controllers trained on 100-150 minutes of data. After 40 minutes, it sustains sub-centimeter mean path error at high operating speeds.