UR-EARL: A framework for designing underwater robots using evolutionary algorithm-driven reinforcement learning

Designing an autonomous underwater vehicle (AUV) is a significant challenge, which involves two primary tasks: shaping the body and developing the control system. Both elements are essential for the AUV’s effective performance. This study introduces a novel framework that integrates evolutionary alg...

Full description

Saved in:
Bibliographic Details
Published in:Ocean engineering Vol. 321; p. 120402
Main Authors: Sariman, Cagatay, Hallawa, Ahmed, Schmeink, Anke
Format: Journal Article
Language:English
Published: Elsevier Ltd 30.03.2025
Subjects:
Autonomous underwater vehicles
Morphological evolution
Reinforcement learning
Robotics
Autonomous underwater vehicles
Morphological evolution
Robotics
Reinforcement learning
ISSN:0029-8018
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Designing an autonomous underwater vehicle (AUV) is a significant challenge, which involves two primary tasks: shaping the body and developing the control system. Both elements are essential for the AUV’s effective performance. This study introduces a novel framework that integrates evolutionary algorithms with reinforcement learning to simultaneously optimize both the shape and behavior of AUVs toward specific objectives. Our framework is designed to be highly expandable, making it easy to incorporate and test new design algorithms alongside existing methods. Additionally, we provide an underwater simulation platform based on the Pybullet engine to realistically evaluate the performance of evolved/trained AUVs. To demonstrate the capabilities of our methodology, we present a case study focused on navigating and maintaining positions within diverse underwater environments with different flow dynamics. We use a Lindenmayer-System-based evolutionary algorithm for optimizing the AUV’s morphology and utilize Stable Baselines 3 algorithms to train the controllers within our proposed framework. Being entirely open-source, we expect this framework to provide researchers with a convenient platform for efficiently testing their innovative methods and accelerating progress in the field. •Concurrent design of the shape and control system of an AUV.•An evolutionary algorithm is used for morphological optimization.•Reinforcement learning is applied for controller training.•A simulator is implemented for realistic simulation of the designed AUVs.
ISSN:0029-8018
DOI:10.1016/j.oceaneng.2025.120402