UR-EARL: A framework for designing underwater robots using evolutionary algorithm-driven reinforcement learning

Designing an autonomous underwater vehicle (AUV) is a significant challenge, which involves two primary tasks: shaping the body and developing the control system. Both elements are essential for the AUV’s effective performance. This study introduces a novel framework that integrates evolutionary alg...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Ocean engineering Ročník 321; s. 120402
Hlavní autori: Sariman, Cagatay, Hallawa, Ahmed, Schmeink, Anke
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Elsevier Ltd 30.03.2025
Predmet:
Autonomous underwater vehicles
Morphological evolution
Reinforcement learning
Robotics
Autonomous underwater vehicles
Morphological evolution
Robotics
Reinforcement learning
ISSN:0029-8018
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Designing an autonomous underwater vehicle (AUV) is a significant challenge, which involves two primary tasks: shaping the body and developing the control system. Both elements are essential for the AUV’s effective performance. This study introduces a novel framework that integrates evolutionary algorithms with reinforcement learning to simultaneously optimize both the shape and behavior of AUVs toward specific objectives. Our framework is designed to be highly expandable, making it easy to incorporate and test new design algorithms alongside existing methods. Additionally, we provide an underwater simulation platform based on the Pybullet engine to realistically evaluate the performance of evolved/trained AUVs. To demonstrate the capabilities of our methodology, we present a case study focused on navigating and maintaining positions within diverse underwater environments with different flow dynamics. We use a Lindenmayer-System-based evolutionary algorithm for optimizing the AUV’s morphology and utilize Stable Baselines 3 algorithms to train the controllers within our proposed framework. Being entirely open-source, we expect this framework to provide researchers with a convenient platform for efficiently testing their innovative methods and accelerating progress in the field. •Concurrent design of the shape and control system of an AUV.•An evolutionary algorithm is used for morphological optimization.•Reinforcement learning is applied for controller training.•A simulator is implemented for realistic simulation of the designed AUVs.
ISSN:0029-8018
DOI:10.1016/j.oceaneng.2025.120402