Steeract improves robot actions from few trial runs without retraining
SteerAct: Unlocking Action Models through Few-Shot Temporal Steering
Robotics
Summary
Robots often struggle to improve their actions quickly without lots of new learning. The paper introduces SteerAct, a way to gently adjust a robot's internal features during its task by watching just a few examples. This adjustment helps the robot fix mistakes on the fly without changing its core program. It works by spotting key signal patterns tied to success or failure and tweaking these at the right times. This method raised robot task success by about 10% in tests without needing to relearn or update the robot’s main model.
What this means in practice
- •For robotics engineers: Improve robot manipulation tasks in real time using minimal new trial data without retraining core models.
- •For industrial automation teams: Quickly enhance robot task success in manufacturing setups by tuning internal feature activations based on small sets of feedback rollouts.
Authors
Kaichen Zhou, Zhiyan Li, Chongkai Gao, Yuhan Wang, Wenlong Huang, Maggie Wang, Fangneng Zhan, Manling Li, Qianqian Wang, Mac Schwager, Zhuang Liu, Li Fei-Fei, Mengyu Wang, Paul Pu Liang
Abstract
Improving pretrained robot action models at test time from only a few rollouts remains challenging. Activation steering offers a lightweight approach by modifying feature activations while keeping model weights fixed. However, effective steering requires identifying which features to modify and determining the timing and strength of modification. We introduce SteerAct, a few-shot temporal steering framework that uses a shared temporal descriptor of sparse-feature dynamics for offline feature selection and online control. A pretrained sparse autoencoder provides candidate features, and our temporal descriptor summarizes the distribution of local activation-order patterns for each feature, enabling comparisons across rollouts without requiring timestep alignment. Before steering begins (offline), we use successful and failed reference rollouts to select a single task-specific feature whose temporal descriptors best distinguish the two outcomes and construct its reference banks. During each new execution (online), we compare the feature's evolving temporal descriptor with these references to determine when to intervene and how strongly to increase or decrease its activation via a steering coefficient. Using only 24 outcome-labeled rollouts per task, SteerAct improves task success by approximately 10% relative to unsteered models in evaluations spanning $π_{0.5}$, Fast-WAM, RoboTwin 2.0, DexJoCo, and real-world manipulation, without updating action-model weights. Code and demos are available at: https://steeract.github.io/.