Optimal Time-Varying Q-Learning Algorithm for Affine Nonlinear Systems With Coupled Players

To address the finite-horizon coupled two-player mixed <inline-formula> <tex-math notation="LaTeX">H_{2}/H_{\infty } </tex-math></inline-formula> control challenge within a continuous-time affine nonlinear system, this research introduces a distinctive Q-function an...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on systems, man, and cybernetics. Systems Vol. 55; no. 10; pp. 7037 - 7047
Main Authors: Zhang, Huaguang, Yu, Shuhang, Sun, Jiayue, Li, Mei
Format: Journal Article
Language:English
Published: IEEE 01.10.2025
Subjects:
Adaptive dynamic programming (ADP)
Couplings
Differential games
finite-horizon
Games
Heuristic algorithms
mixed H₂/H∞ control
Nash equilibrium
neural network (NN)
Optimal control
Q-learning
System dynamics
Time-varying systems
ISSN:2168-2216, 2168-2232
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:To address the finite-horizon coupled two-player mixed <inline-formula> <tex-math notation="LaTeX">H_{2}/H_{\infty } </tex-math></inline-formula> control challenge within a continuous-time affine nonlinear system, this research introduces a distinctive Q-function and presents an innovative adaptive dynamic programming (ADP) method that operates autonomously of system-specific information. Initially, we formulate the time-varying Hamilton-Jacobi-Isaacs (HJI) equations, which pose a significant challenge for resolution due to their time-dependent and nonlinear nature. Subsequently, a novel offline policy iteration (PI) algorithm is introduced, highlighting its convergence and reinforcing the substantive proof of the existence of Nash equilibrium points. Moreover, a novel action-dependent Q-function is established to facilitate entirely model-free learning, representing the initial foray into the mixed <inline-formula> <tex-math notation="LaTeX">H_{2}/H_{\infty } </tex-math></inline-formula> control problem involving coupled players. The Lyapunov direct approach is employed to ensure the stability of the closed-loop uncertain affine nonlinear system under the ADP-based control scheme, guaranteeing uniform ultimate boundedness (UUB). Finally, a numerical simulation is conducted to validate the effectiveness of the aforementioned ADP-based control approach.
ISSN:2168-2216
2168-2232
DOI:10.1109/TSMC.2025.3580988