Decentralized learning improves tugboat teamwork for better ship control
SMaRT-Tug: Structured Multi-Agent Reinforcement Learning for Physics-Based Tugboat-Barge Collaborative Manipulation
Robotics
Summary
Controlling multiple tugboats to move large ships is tricky because of water forces, waves, and the way tugboats push. The authors created a detailed computer simulator that models real water physics to train tugboats to work together better. They used a special learning technique that helps the tugboats learn how to push accurately and safely. Their system works well even when facing rough seas or when more tugboats join the team, without needing extra training.
multi-agent reinforcement learningtugboat maneuveringhydrodynamicsdecentralized controlphysics-based simulationbuoyancywave modelingpolicy optimizationzero-shot generalizationunderactuated dynamics
Authors
Junkai Lu, Jiadong Zhao, Jiacheng Zhang, Wenqi Zhao, Hao Gen Chia, Qun Shen Png, Germaine Ee, Chengyang He, Yifeng Zhang, Nathanael Tan, Guillaume Sartoretti
Abstract
Autonomous tugboating is central for automating maritime operations such as port logistics and vessel maneuvering, where multiple tugboats must cooperatively transport/manipulate a larger vessel. Collaborative pushing in this setting is challenging due to coupled hydrodynamics, low resistance, strong environmental disturbances, underactuated barge dynamics, and contact-rich interactions. Conventional control methods often rely on simplified models and fixed configurations, which limit their adaptability, while learning-based approaches are constrained by the lack of scalable and physically realistic training environments. We address these challenges by introducing a physics-based, GPU-accelerated simulation and learning framework for collaborative tugboat manipulation. Our simulator incorporates a customized buoyancy model, wave modeling, and hydrodynamic resistance, and supports large-scale multi-agent training under marine dynamics. In this simulator, we train a decentralized MAPPO (Multi-Agent PPO) policy augmented with a structured control prior (SCP) to improve training stability and maintain feasible pushing configurations. We evaluate our learned policy on straight-line transit, turning, and deceleration tasks, where we show that our decentralized framework yields more reliable and accurate maneuvering performance compared to a PID-based controller and a centralized PPO baseline. We further demonstrate zero-shot generalization to more challenging sea states and advanced maneuvers, as well as zero-shot scalability to larger teams of three and four tugboats despite training with only two agents.