Twin Deterministic Policy Gradient Adaptive Dynamic Programming for Optimal Control of Affine Nonlinear Discrete-time Systems

Recent achievements in the field of adaptive dynamic programming (ADP), as well as the data resources and computational capabilities in modern control systems, have led to a growing interest in learning and data-driven control technologies. This paper proposes a twin deterministic policy gradient ad...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:International journal of control, automation, and systems Ročník 20; číslo 9; s. 3098 - 3109
Hlavní autori: Xu, Jiahui, Wang, Jingcheng, Rao, Jun, Zhong, Yanjiu, Zhao, Shangwei
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Bucheon / Seoul Institute of Control, Robotics and Systems and The Korean Institute of Electrical Engineers 01.09.2022
Springer Nature B.V
제어·로봇·시스템학회
Predmet:
Adaptive control
Algorithms
Control
Discrete time systems
Dynamic programming
Engineering
Liapunov functions
Machine learning
Mechatronics
Nonlinear control
Nonlinear systems
Optimal control
Regular Paper
Robotics
제어계측공학
optimal control
affine nonlinear system
twin deterministic policy gradient
Adaptive dynamic programming
ISSN:1598-6446, 2005-4092
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Recent achievements in the field of adaptive dynamic programming (ADP), as well as the data resources and computational capabilities in modern control systems, have led to a growing interest in learning and data-driven control technologies. This paper proposes a twin deterministic policy gradient adaptive dynamic programming (TDPGADP) algorithm to solve the optimal control problem for a discrete-time affine nonlinear system in a model-free scenario. To solve the overestimation problem resulted from function approximation errors, the minimum value between the double Q network is taken to update the control policy. The convergence of the proposed algorithm in which the value function is served as the Lyapunov function is verified. By designing a twin actor-critic network structure, combining the target network and a specially designed adaptive experience replay mechanism, the algorithm is convenient to implement and the sample efficiency of the learning process can be improved. Two simulation examples are conducted to verify the efficacy of the proposed method.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
http://link.springer.com/article/10.1007/s12555-021-0473-6
ISSN:1598-6446
2005-4092
DOI:10.1007/s12555-021-0473-6