Complete stability analysis of a heuristic approximate dynamic programming control design

This paper provides new stability results for Action-Dependent Heuristic Dynamic Programming (ADHDP), using a control algorithm that iteratively improves an internal model of the external world in the autonomous system based on its continuous interaction with the environment. We extend previous resu...

Full description

Saved in:
Bibliographic Details
Published in:Automatica (Oxford) Vol. 59; pp. 9 - 18
Main Authors: Sokolov, Yury, Kozma, Robert, Werbos, Ludmilla D., Werbos, Paul J.
Format: Journal Article
Language:English
Published: Elsevier Ltd 01.09.2015
Subjects:
Action-Dependent Heuristic Dynamic Programming
Adaptive control
Adaptive critic
Adaptive dynamic programming
Gradient descent
Lyapunov function
Neural network
Gradient descent
Action-Dependent Heuristic Dynamic Programming
Adaptive dynamic programming
Adaptive critic
Neural network
Adaptive control
Lyapunov function
ISSN:0005-1098, 1873-2836
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper provides new stability results for Action-Dependent Heuristic Dynamic Programming (ADHDP), using a control algorithm that iteratively improves an internal model of the external world in the autonomous system based on its continuous interaction with the environment. We extend previous results for ADHDP control to the case of general multi-layer neural networks with deep learning across all layers. In particular, we show that the introduced control approach is uniformly ultimately bounded (UUB) under specific conditions on the learning rates, without explicit constraints on the temporal discount factor. We demonstrate the benefit of our results to the control of linear and nonlinear systems, including the cart–pole balancing problem. Our results show significantly improved learning and control performance as compared to the state-of-the-art.
ISSN:0005-1098
1873-2836
DOI:10.1016/j.automatica.2015.06.001