Robot adapts thinking time to improve real-time control

Learning Robot Metareasoning: Adaptive Computation Budgets for Real-Time Control

Robotics

Summary

Planning robot actions requires balancing between thinking time to choose good actions and reacting quickly to changes. The authors show that instead of fixing how long a robot spends planning, the robot can learn when to plan longer or shorter based on the situation it faces. They teach the robot this by using reinforcement learning, which lets the robot figure out the best balance in real time. Their tests on robots that navigate, fly, and handle objects show this method works better than using a fixed planning time.

What this means in practice

  • For robotics software developers: Design control software that dynamically adjusts planning time to improve responsiveness and safety in navigation and manipulation tasks.
  • For autonomous drone operators: Enable quadrotor flight systems to allocate computation time adaptively, improving flight agility and handling of complex environments.

Authors

Lorenzo Magnino, Antonio Marino, Michael Amir, Daigo Shishika, Amanda Prorok

Abstract

Planning while moving faces an historical trade-off between deliberation and reactivity. A planner running concurrently with execution needs time to compute, so the robot must commit to an open-loop action chunk for that duration: a longer commitment buys a better plan at the cost of feedback frequency. Existing controllers fix this action commitment, and consequently the computation budget, before deployment. However, the optimal computation time is highly state-dependent (due to factors such as unmodeled dynamics, safety constraints, and marginal optimisation gains), i.e. dynamic, therefore offline static tuning risks leaving systems blind during critical maneuvers or unable to find global effective solutions. We cast the identification of the optimal computation budget as a semi-Markov decision process (SMDP) in which a single decision defines both the robot action commitments and the time allocated to the solver. To solve this, we train a reinforcement-learning policy conditioned on the current state, the executing plan, and inexpensive planner statistics. Across ground-robot navigation, agile quadrotor flight, and contact-rich manipulation, the learned policy changes its budget with the demands of the scenario, discovering efficient reasoning patterns that outperform fixed-budget baselines. https://lorenzomagnino.github.io/learning-robot-metareasoning-review/