Six degree freedom underwater vehicle control matches classical and learning methods
A Unified Dynamics Framework for Reinforcement Learning and Classical Control of a Six-DOF Pipeline-Tracking ROV in NVIDIA Isaac Sim
RoboticsArtificial Intelligence
Summary
Controlling underwater drones is tricky because computer programs often learn in one setting but work differently in real life. The authors made a system where the same physics rules control both a simulated underwater vehicle and a training program. They compared several smart learning techniques and traditional control methods for guiding an underwater robot along pipelines. Some learning approaches and a classical math method worked best, while others had specific problems. This shows that carefully matching physics in simulations helps compare different control ways more fairly.
What this means in practice
- •For marine robotics engineers: Use a unified physics-based simulation to train and evaluate underwater vehicle navigation controllers with consistent real-world parameters.
- •For autonomous vehicle developers: Benchmark reinforcement learning and classical control methods under identical dynamics models for more reliable underwater pipeline tracking.
Authors
Cheng Siong Chin, M. Venkateshkumar, Jianhua Zhang
Abstract
Reinforcement learning controllers for underwater vehicles are usually trained against one physics representation and deployed against another, so reported performance does not always describe behavior outside training. This paper presents a pipeline-tracking architecture for a six-degree-of-freedom remotely operated vehicle (ROV) in which one Universal Scene Description (USD) scene supplies the real BlueROV2-Heavy mass, added-mass, damping, buoyancy, and thruster parameters to both halves of the system: a vectorized NumPy implementation of Fossen's marine-craft equations, and an interactive NVIDIA Isaac Sim deployment applying the identical equations as PhysX forces at every step. The Coriolis-centripetal term is the primary dynamics model in both branches; a controlled ablation on PPO and TRPO shows that including it does not destabilize either algorithm and modestly improves tracking, about 19 percent lower standoff RMS error for TRPO. Five reinforcement learning algorithms, PPO, soft actor-critic, TD3, DDPG, and TRPO, are trained against one environment, reward, and randomized evaluation harness through a checkpoint-compatibility layer scoring any policy with the same code. The pipeline is extended with six classical baselines, PID, sliding-mode, fuzzy logic, feedback linearization, model predictive control, and an adaptive neuro-fuzzy inference system, driven by the same guidance geometry and thruster allocation as the learned policies. Under Coriolis-enabled dynamics, PPO, TRPO, and feedback linearization reach the strongest combination of 100 percent success and competitive accuracy; PID, fuzzy control, and the neuro-fuzzy baseline also reach 100 percent success with looser tracking; DDPG and TD3 each show a specific, explainable failure mode rather than a general weakness of off-policy learning; and classical control remains a strong baseline against the best learned policies.