UR-EARL: A framework for designing underwater robots using evolutionary algorithm-driven reinforcement learning

Designing an autonomous underwater vehicle (AUV) is a significant challenge, which involves two primary tasks: shaping the body and developing the control system. Both elements are essential for the AUV’s effective performance. This study introduces a novel framework that integrates evolutionary alg...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Ocean engineering Jg. 321; S. 120402
Hauptverfasser: Sariman, Cagatay, Hallawa, Ahmed, Schmeink, Anke
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Elsevier Ltd 30.03.2025
Schlagworte:
Autonomous underwater vehicles
Morphological evolution
Reinforcement learning
Robotics
Autonomous underwater vehicles
Morphological evolution
Robotics
Reinforcement learning
ISSN:0029-8018
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Designing an autonomous underwater vehicle (AUV) is a significant challenge, which involves two primary tasks: shaping the body and developing the control system. Both elements are essential for the AUV’s effective performance. This study introduces a novel framework that integrates evolutionary algorithms with reinforcement learning to simultaneously optimize both the shape and behavior of AUVs toward specific objectives. Our framework is designed to be highly expandable, making it easy to incorporate and test new design algorithms alongside existing methods. Additionally, we provide an underwater simulation platform based on the Pybullet engine to realistically evaluate the performance of evolved/trained AUVs. To demonstrate the capabilities of our methodology, we present a case study focused on navigating and maintaining positions within diverse underwater environments with different flow dynamics. We use a Lindenmayer-System-based evolutionary algorithm for optimizing the AUV’s morphology and utilize Stable Baselines 3 algorithms to train the controllers within our proposed framework. Being entirely open-source, we expect this framework to provide researchers with a convenient platform for efficiently testing their innovative methods and accelerating progress in the field. •Concurrent design of the shape and control system of an AUV.•An evolutionary algorithm is used for morphological optimization.•Reinforcement learning is applied for controller training.•A simulator is implemented for realistic simulation of the designed AUVs.
ISSN:0029-8018
DOI:10.1016/j.oceaneng.2025.120402