Structured multi-agent learning improves autonomous tugboat and barge control
SMaRT-Tug: Structured Multi-Agent Reinforcement Learning for Physics-Based Tugboat-Barge Collaborative Manipulation
Robotics
Summary
Controlling tugboats to push and move large barges is tricky because of water forces, waves, and the way the barge moves. The authors created a detailed computer simulation that mimics how tugboats and barges behave in real water, including waves and resistance. Using this simulator, they trained multiple tugboats to cooperate effectively using a new learning method, achieving better control than traditional methods. Their approach works well even when facing harder sea conditions or when using more tugboats than initially trained for.
What this means in practice
- •For maritime automation engineers: Develop autonomous tugboat systems capable of coordinated barge maneuvering under realistic sea conditions using learned multi-agent controllers.
- •For robotics software developers: Use GPU-accelerated physics-based simulators to train and test multi-agent control policies for complex marine vehicle interactions.
Authors
Junkai Lu, Jiadong Zhao, Jiacheng Zhang, Wenqi Zhao, Hao Gen Chia, Qun Shen Png, Germaine Ee, Chengyang He, Yifeng Zhang, Nathanael Tan, Guillaume Sartoretti
Abstract
Autonomous tugboating is central for automating maritime operations such as port logistics and vessel maneuvering, where multiple tugboats must cooperatively transport/manipulate a larger vessel. Collaborative pushing in this setting is challenging due to coupled hydrodynamics, low resistance, strong environmental disturbances, underactuated barge dynamics, and contact-rich interactions. Conventional control methods often rely on simplified models and fixed configurations, which limit their adaptability, while learning-based approaches are constrained by the lack of scalable and physically realistic training environments. We address these challenges by introducing a physics-based, GPU-accelerated simulation and learning framework for collaborative tugboat manipulation. Our simulator incorporates a customized buoyancy model, wave modeling, and hydrodynamic resistance, and supports large-scale multi-agent training under marine dynamics. In this simulator, we train a decentralized MAPPO (Multi-Agent PPO) policy augmented with a structured control prior (SCP) to improve training stability and maintain feasible pushing configurations. We evaluate our learned policy on straight-line transit, turning, and deceleration tasks, where we show that our decentralized framework yields more reliable and accurate maneuvering performance compared to a PID-based controller and a centralized PPO baseline. We further demonstrate zero-shot generalization to more challenging sea states and advanced maneuvers, as well as zero-shot scalability to larger teams of three and four tugboats despite training with only two agents.