UR-EARL: A framework for designing underwater robots using evolutionary algorithm-driven reinforcement learning - Sariman, Cagatay; Hallawa, Ahmed; Schmeink, Anke (Ocean engineering, 2025)
Designing an autonomous underwater vehicle (AUV) is a significant challenge, which involves two primary tasks: shaping the body and developing the control system. Both elements are essential for the AUV’s effective performance. This study introduces a novel framework that integrates evolutionary algorithms with reinforcement learning to simultaneously optimize both the shape and behavior of AUVs toward specific objectives. Our framework is designed to be highly expandable, making it easy to incorporate and test new design algorithms alongside existing methods. Additionally, we provide an underwater simulation platform based on the Pybullet engine to realistically evaluate the performance of evolved/trained AUVs. To demonstrate the capabilities of our methodology, we present a case study focused on navigating and maintaining positions within diverse underwater environments with different flow dynamics. We use a Lindenmayer-System-based evolutionary algorithm for optimizing the AUV’s morphology and utilize Stable Baselines 3 algorithms to train the controllers within our proposed framework. Being entirely open-source, we expect this framework to provide researchers with a convenient platform for efficiently testing their innovative methods and accelerating progress in the field. •Concurrent design of the shape and control system of an AUV.•An evolutionary algorithm is used for morphological optimization.•Reinforcement learning is applied for controller training.•A simulator is implemented for realistic simulation of the designed AUVs.