Adaptive Q-Learning Based Model-Free H Control of Continuous-Time Nonlinear Systems: Theory and Application

Although model based <inline-formula><tex-math notation="LaTeX">H_{\infty }</tex-math></inline-formula> control scheme for nonlinear continuous-time (CT) systems with unknown system dynamics has been extensively studied, model-free <inline-formula><tex-math...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on emerging topics in computational intelligence Vol. 9; no. 2; pp. 1143 - 1152
Main Authors: Zhao, Jun, Lv, Yongfeng, Wang, Zhangu, Zhao, Ziliang
Format: Journal Article
Language:English
Published: IEEE 01.04.2025
Subjects:
<named-content xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ali="http://www.niso.org/schemas/ali/1.0/" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" content-type="math" xlink:type="simple"> <inline-formula> <tex-math notation="LaTeX"> H_{\infty }</tex-math> </inline-formula> </named-content> control
Artificial neural networks
Control systems
Cost function
Heuristic algorithms
learning law
Nonlinear systems
Q-learning
Reinforcement learning
System dynamics
ISSN:2471-285X, 2471-285X
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Although model based <inline-formula><tex-math notation="LaTeX">H_{\infty }</tex-math></inline-formula> control scheme for nonlinear continuous-time (CT) systems with unknown system dynamics has been extensively studied, model-free <inline-formula><tex-math notation="LaTeX">H_{\infty }</tex-math></inline-formula> control of nonlinear CT systems via Q-learning is still a challenging problem. This paper develops a novel Q-learning based model-free <inline-formula><tex-math notation="LaTeX">H_{\infty }</tex-math></inline-formula> control scheme for nonlinear CT systems, where the adaptive critic and actor continuously and simultaneously update each other, eliminating the need for iterative steps. As a result, a hybrid structure is avoided and there is no longer a requirement for an initial stabilizing control policy. To obtain the <inline-formula><tex-math notation="LaTeX">H_{\infty }</tex-math></inline-formula> control of the CT nonlinear system, the Q-learning strategy is introduced to online resolve the <inline-formula><tex-math notation="LaTeX">H_{\infty }</tex-math></inline-formula> control problem in a non-iterative approach, where the system dynamics are not required. In addition, a new learning law is further developed by utilizing a sliding mode scheme to online update the critic neural network (NN) weights. Due to the strong convergence of critic NN weights, the actor NN used in most <inline-formula><tex-math notation="LaTeX">H_{\infty }</tex-math></inline-formula> control algorithms is removed. Finally, numerical simulation and experimental results of an adaptive cruise control (ACC) system based on a real vehicle effectively demonstrate the feasibility of the presented control method and learning algorithm.
ISSN:2471-285X
2471-285X
DOI:10.1109/TETCI.2024.3449870