Adaptive Q-Learning Based Model-Free H Control of Continuous-Time Nonlinear Systems: Theory and Application

Although model based <inline-formula><tex-math notation="LaTeX">H_{\infty }</tex-math></inline-formula> control scheme for nonlinear continuous-time (CT) systems with unknown system dynamics has been extensively studied, model-free <inline-formula><tex-math...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on emerging topics in computational intelligence Jg. 9; H. 2; S. 1143 - 1152
Hauptverfasser: Zhao, Jun, Lv, Yongfeng, Wang, Zhangu, Zhao, Ziliang
Format: Journal Article
Sprache:Englisch
Veröffentlicht: IEEE 01.04.2025
Schlagworte:
<named-content xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ali="http://www.niso.org/schemas/ali/1.0/" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" content-type="math" xlink:type="simple"> <inline-formula> <tex-math notation="LaTeX"> H_{\infty }</tex-math> </inline-formula> </named-content> control
Artificial neural networks
Control systems
Cost function
Heuristic algorithms
learning law
Nonlinear systems
Q-learning
Reinforcement learning
System dynamics
ISSN:2471-285X, 2471-285X
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Although model based <inline-formula><tex-math notation="LaTeX">H_{\infty }</tex-math></inline-formula> control scheme for nonlinear continuous-time (CT) systems with unknown system dynamics has been extensively studied, model-free <inline-formula><tex-math notation="LaTeX">H_{\infty }</tex-math></inline-formula> control of nonlinear CT systems via Q-learning is still a challenging problem. This paper develops a novel Q-learning based model-free <inline-formula><tex-math notation="LaTeX">H_{\infty }</tex-math></inline-formula> control scheme for nonlinear CT systems, where the adaptive critic and actor continuously and simultaneously update each other, eliminating the need for iterative steps. As a result, a hybrid structure is avoided and there is no longer a requirement for an initial stabilizing control policy. To obtain the <inline-formula><tex-math notation="LaTeX">H_{\infty }</tex-math></inline-formula> control of the CT nonlinear system, the Q-learning strategy is introduced to online resolve the <inline-formula><tex-math notation="LaTeX">H_{\infty }</tex-math></inline-formula> control problem in a non-iterative approach, where the system dynamics are not required. In addition, a new learning law is further developed by utilizing a sliding mode scheme to online update the critic neural network (NN) weights. Due to the strong convergence of critic NN weights, the actor NN used in most <inline-formula><tex-math notation="LaTeX">H_{\infty }</tex-math></inline-formula> control algorithms is removed. Finally, numerical simulation and experimental results of an adaptive cruise control (ACC) system based on a real vehicle effectively demonstrate the feasibility of the presented control method and learning algorithm.
ISSN:2471-285X
2471-285X
DOI:10.1109/TETCI.2024.3449870