Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning

In this paper, a new formulation for the optimal tracking control problem (OTCP) of continuous-time nonlinear systems is presented. This formulation extends the integral reinforcement learning (IRL) technique, a method for solving optimal regulation problems, to learn the solution to the OTCP. Unlik...

Full description

Saved in:
Bibliographic Details
Published in:Automatica (Oxford) Vol. 50; no. 7; pp. 1780 - 1792
Main Authors: Modares, Hamidreza, Lewis, Frank L.
Format: Journal Article
Language:English
Published: Kidlington Elsevier Ltd 01.07.2014
Elsevier
Subjects:
Algorithmics. Computability. Computer arithmetics
Applied sciences
Artificial intelligence
Computer science; control theory; systems
Control system synthesis
Control theory. Systems
Drift
Dynamical systems
Dynamics
Exact sciences and technology
Input constrainers
Integral reinforcement learning
Mathematical analysis
Neural networks
Nonlinear dynamics
Optimal control
Optimal tracking control
Optimization
Reinforcement
Theoretical computing
Tracking
Optimal tracking control
Integral reinforcement learning
Neural networks
Input constrainers
Unknown input
Tracking
Non linear control
Control synthesis
Continuous time
Online algorithm
Dynamical system
Control constraint
Uncertain system
Optimal control (mathematics)
Mathematical programming
Bellman equation
Stability
Reinforcement learning
Tracking task
Neural network
Non linear system
Constrained optimization
Persistence
Hamilton Jacobi equation
Optimal control
Optimal solution
Problem solving
ISSN:0005-1098, 1873-2836
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this paper, a new formulation for the optimal tracking control problem (OTCP) of continuous-time nonlinear systems is presented. This formulation extends the integral reinforcement learning (IRL) technique, a method for solving optimal regulation problems, to learn the solution to the OTCP. Unlike existing solutions to the OTCP, the proposed method does not need to have or to identify knowledge of the system drift dynamics, and it also takes into account the input constraints a priori. An augmented system composed of the error system dynamics and the command generator dynamics is used to introduce a new nonquadratic discounted performance function for the OTCP. This encodes the input constrains into the optimization problem. A tracking Hamilton–Jacobi–Bellman (HJB) equation associated with this nonquadratic performance function is derived which gives the optimal control solution. An online IRL algorithm is presented to learn the solution to the tracking HJB equation without knowing the system drift dynamics. Convergence to a near-optimal control solution and stability of the whole system are shown under a persistence of excitation condition. Simulation examples are provided to show the effectiveness of the proposed method.
Bibliography:ObjectType-Article-2
SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 23
ObjectType-Article-1
ObjectType-Feature-2
ISSN:0005-1098
1873-2836
DOI:10.1016/j.automatica.2014.05.011