Autoregressive robot policies improve action prediction with self-correcting tokens
ReAT: Residual Action Tokenization for Autoregressive Robot Policies
Robotics
Summary
Robots often plan their movements by breaking actions into smaller steps called tokens. Usually, these tokens don’t clearly show how each one helps build the final movement. The researchers created a new method called Residual Action Tokenization (ReAT) where each token fixes errors left by the previous ones, improving the robot’s action plans. Tests showed this method works better than older ones on many tasks, even using fewer tokens. This approach helps robots make more accurate and refined movements over time.
What this means in practice
- •For robotic system developers: Improve robot manipulators’ action planning by integrating residual token updates that reduce prediction errors stepwise.
- •For industrial automation teams: Enhance precision in robotic assembly and handling tasks by employing tokens that iteratively refine robot actions.
Authors
Dijia Zhan, Zhuo Li, Xuemiao Xu, Sun Zheng, Jinyi Li, Wenjun Wang, Jie Tang
Abstract
Autoregressive robot policies rely on action tokenizers to convert continuous action chunks into compact sequences of discrete prediction targets. Existing tokenizers can reconstruct action chunks from discrete codes, but often leave implicit what each individual token contributes to the reconstructed action. We introduce Residual Action Tokenization (ReAT), which gives each token an explicit role in refining the current action reconstruction. It uses the decoded reconstruction as feedback to construct the next token from the remaining action error. This closes the loop between encoding and decoding, making each new token responsive to the errors left by the actual decoded prefix. Across 19 manipulation tasks spanning three simulation benchmarks and real-world settings, 8-token ReAT policies outperform the evaluated baselines in task success; on LIBERO, even 4-token ReAT surpasses the evaluated 8-token action-tokenization baselines at half the token budget. These results suggest a path toward action tokenization that supports self-corrective policy learning, helping autoregressive policies learn to progressively refine their own action plans.