Action chunking improves robot control with quick online corrections
Action Chunking Proximal Policy Optimization with Feedback Correction
Machine LearningRobotics
Summary
Robots often need to make dozens of tiny moves quickly, which can be slow or hard to manage. The authors show a way to group these moves into small sets (chunks) so robots can plan more easily without losing the ability to react to changes. They add a correction step that adjusts each move inside a chunk as the robot acts, making control smarter and more flexible. Testing on many simulated robot tasks shows this method works better than other approaches.
What this means in practice
- •For robotics engineers: Improve control algorithms for robots handling complex physical tasks by combining action planning and fast feedback adjustments.
- •For video game developers: Implement more lifelike and responsive control of virtual characters by using chunked action planning with in-game feedback correction.
Authors
Sanghyun Hahn, Jonghyun Choi
Abstract
Action chunking provides temporal abstraction in reinforcement learning by selecting short action sequences instead of individual actions, but many existing approaches face two limitations in high-dimensional robotic control. First, many rely on value functions over action chunks, which can be difficult to learn as action dimensionality and chunk length grow. Second, executing chunks open-loop removes within-chunk feedback, limiting reactivity in contact-rich tasks. We present Action Chunking PPO (ACPPO), a PPO extension that uses a chunked actor while retaining a standard state-value critic, thereby avoiding chunked Q-functions. We further propose ACPPO-Corr, which augments the chunk planner with a stepwise feedback corrector that adjusts planned actions online within each chunk. Across 25 simulated robotics tasks from IsaacGym and Bi-DexHands, spanning locomotion, arm manipulation, and dexterous hand-object interaction, ACPPO-Corr achieves the strongest aggregate performance among evaluated methods and performs best on both decision-frequency-sensitive and decision-frequency-neutral task subsets. Ablations show that moderate chunk lengths work best and that corrector regularization is important for balancing chunk-level planning with local feedback. These results suggest that action chunking can be effective in online PPO when chunk-level planning is paired with closed-loop correction. The code is available at: https://github.com/hshhahn/ACPPO.