Action chunking transformer encoder removal impact revisited in robot learning
The Latent That Never Was: A Forensic Re-run of the CVAE Ablation in Action Chunking Transformer
RoboticsMachine Learning
Summary
The paper re-examines a key claim about robot learning models called Action Chunking Transformers (ACT). The original claim said removing part of the model called the encoder made the robot much worse at completing tasks. The authors reran tests and found this big drop did not reappear, suggesting the encoder might not be as crucial as thought. They also found the encoder’s internal information is not really used during task execution, and skipping it speeds up training. They share their code so others can check and explore further.
What this means in practice
- •For robot software engineers: Improve training speed in robot manipulation models by removing unused encoder components without major performance loss.
- •For machine learning practitioners: Use the released code and evaluation tools to verify model components’ usefulness in similar tasks and datasets.
Authors
Bo Kang
Abstract
Action Chunking Transformers (ACT) are widely used to learn robot manipulation from demonstrations. Their conditional variational autoencoder includes an encoder meant to capture differences between demonstrations during training. The original ACT paper reported that encoder removal dropped the mean success rate from 35% to 2% on two simulated tasks with human demonstrations. We re-ran this ablation in the original code and checked whether the findings depend on the implementation or training data. The published drop does not reappear in our tests, although smaller gains or losses in success rate remain uncertain. To investigate the discrepancy, we varied training length and how checkpoints are selected for evaluation. Both can reverse which policy scores higher, but the published drop's cause remains unknown. Success rates alone leave open whether the encoder provides information that helps the policy reconstruct demonstrated actions. On the tested ACT benchmark, the sampled latent provides little reconstruction benefit at every tested nonzero weight of the penalty on latent information. At inference, ACT leaves this latent unused and sets it to zero. Skipping the encoder increases training throughput in both implementations we timed. We release code, evaluation tools and results so others can repeat the comparisons and test the encoder on other tasks.