Soft actor critic outperforms ppo in dual arm space robot control
PPO vs. SAC: A Comparative Study for Cooperative Motion Control of Dual-Arm Space Robots
Artificial Intelligence
Summary
Space robots with two arms can better handle tasks like grabbing objects in space, but controlling both arms smoothly without collisions is tricky. The authors compared two advanced AI learning methods, PPO and SAC, to teach a simulated dual-arm space robot to reach targets accurately while avoiding collisions. They found SAC learned faster, was more precise, and worked better on new, unseen tasks. This shows SAC may be a more reliable method for coordinating complex space robot movements.
What this means in practice
- •For space robotics engineers: Design cooperative control systems for dual-arm space robots to improve grasping and manipulation near orbital debris or satellites.
- •For industrial robot controllers: Enhance coordinated motion planning for dual-arm robots in factory settings using SAC to reduce collisions and improve task synchronization.
Authors
Nasim Fallahiarezoodar, Zheng H. Zhu
Abstract
The growing population of orbital debris has intensified interest in active debris removal and on-orbit servicing, for which dual-arm space manipulators offer improved grasping stability and load distribution relative to single-arm systems. However, coordinating two manipulators under inter-arm collision avoidance, orientation alignment, and sensing uncertainty remains challenging. This paper develops a coordinated reinforcement learning framework for cooperative motion planning of a dual-arm space manipulator in a target-reaching task with inter-arm collision-avoidance constraints. A centralized state representation combines joint-space, end-effector, and target-relative information, while a multi-objective reward function promotes target-reaching accuracy, orientation alignment, motion smoothness, synchronized task completion, and collision avoidance. Within this framework, Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC) are compared as representative on-policy and off-policy algorithms, and the learned policies are further evaluated for generalization to previously unseen target configurations. Across five random seeds, SAC substantially outperforms PPO within the considered fixed-base simulation scenarios: it reaches an 80% success threshold in every seed (median 1.6M environment steps) whereas PPO does not reach it within a 4M-step budget, and it attains a higher mean success rate (78.6% versus 7.4%) with lower position and orientation errors and tighter bimanual synchronization. Both algorithms degrade on unseen boundary targets, but SAC retains most of its performance while PPO's collapses. The results also expose a failure mode of PPO: under the two-phase shaped reward, it tends to converge to a reward-loitering local optimum rather than committing to the precision task. This behavior highlights the sensitivity of on-policy learning to reward shaping.