Novel Discounted Adaptive Critic Control Designs With Accelerated Learning Formulation

Inspired by the successive relaxation method, a novel discounted iterative adaptive dynamic programming framework is developed, in which the iterative value function sequence possesses an adjustable convergence rate. The different convergence properties of the value function sequence and the stabili...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on cybernetics Vol. 54; no. 5; pp. 1 - 14
Main Authors: Ha, Mingming, Wang, Ding, Liu, Derong
Format: Journal Article
Language:English
Published: United States IEEE 01.05.2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Adaptive control
Adaptive critic designs
adaptive dynamic programming (ADP)
Algorithms
Closed loop systems
Closed loops
Convergence
Cost function
discrete-time nonlinear systems
Dynamic programming
fast convergence rate
Feedback control
Iterative methods
Machine learning
Optimal control
Power system stability
reinforcement learning
Relaxation method (mathematics)
Stability criteria
value iteration (VI)
ISSN:2168-2267, 2168-2275, 2168-2275
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Inspired by the successive relaxation method, a novel discounted iterative adaptive dynamic programming framework is developed, in which the iterative value function sequence possesses an adjustable convergence rate. The different convergence properties of the value function sequence and the stability of the closed-loop systems under the new discounted value iteration (VI) are investigated. Based on the properties of the given VI scheme, an accelerated learning algorithm with convergence guarantee is presented. Moreover, the implementations of the new VI scheme and its accelerated learning design are elaborated, which involve value function approximation and policy improvement. A nonlinear fourth-order ball-and-beam balancing plant is used to verify the performance of the developed approaches. Compared with the traditional VI, the present discounted iterative adaptive critic designs greatly accelerate the convergence rate of the value function and reduce the computational cost simultaneously.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:2168-2267
2168-2275
2168-2275
DOI:10.1109/TCYB.2022.3233593