Optimization of reward shaping function based on genetic algorithm applied to a cross validated deep deterministic policy gradient in a powered landing guidance problem

One major capability of a Deep Reinforcement Learning (DRL) agent to control a specific vehicle in an environment without any prior knowledge is decision-making based on a well-designed reward shaping function. An important but little-studied major factor that can alter significantly the training re...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Engineering applications of artificial intelligence Ročník 120; s. 105798
Hlavní autori: Nugroho, Larasmoyo, Andiarti, Rika, Akmeliawati, Rini, Kutay, Ali Türker, Larasati, Diva Kartika, Wijaya, Sastra Kusuma
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Elsevier Ltd 01.04.2023
Predmet:
DDPG
Fitness
GA-search
Reusable launch vehicle
Reward shaping function
Reward shaping function
DDPG
Reusable launch vehicle
GA-search
Fitness
ISSN:0952-1976, 1873-6769
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:One major capability of a Deep Reinforcement Learning (DRL) agent to control a specific vehicle in an environment without any prior knowledge is decision-making based on a well-designed reward shaping function. An important but little-studied major factor that can alter significantly the training reward score and performance outcomes is the reward shaping function. To maximize the control efficacy of a DRL algorithm, an optimized reward shaping function and a solid hyperparameter combination are essential. In order to achieve optimal control during the powered descent guidance (PDG) landing phase of a reusable launch vehicle, the Deep Deterministic Policy Gradient (DDPG) algorithm is used in this paper to discover the best shape of the reward shaping function (RSF). Although DDPG is quite capable of managing complex environments and producing actions intended for continuous spaces, its state and action performance could still be improved. A reference DDPG agent with the original reward shaping function and a PID controller were placed side by side with the GA-DDPG agent using GA-optimized RSF. The best GA-DDPG individual can maximize overall rewards and minimize state errors with the help of the potential-based GA(PbGA) searched RSF, maintaining the highest fitness score among all individuals after has been cross-validated and retested extensively Monte-Carlo experimental results.
ISSN:0952-1976
1873-6769
DOI:10.1016/j.engappai.2022.105798