Data-driven policy iteration algorithm for optimal control of continuous-time Itô stochastic systems with Markovian jumps

This studies the infinite horizon optimal control problem for a class of continuous-time systems subjected to multiplicative noises and Markovian jumps by using a data-driven policy iteration algorithm. The optimal control problem is equivalent to solve a stochastic coupled algebraic Riccatic equati...

Full description

Saved in:
Bibliographic Details
Published in:IET control theory & applications Vol. 10; no. 12; pp. 1431 - 1439
Main Authors: Song, Jun, He, Shuping, Liu, Fei, Niu, Yugang, Ding, Zhengtao
Format: Journal Article
Language:English
Published: The Institution of Engineering and Technology 08.08.2016
Subjects:
Algorithms
continuous time systems
continuous‐time Itô stochastic systems
convergence of numerical methods
implicit iterative algorithm
infinite horizon optimal control problem
Iterative algorithms
Iterative methods
Markov processes
Markovian jumps
Mathematical analysis
Mathematical models
multiplicative noises
offline iteration algorithm
Optimal control
parallel Kleinman iterative equations
Policies
Riccati equations
Special Issue: Data-driven Control, and Data-Based System Modelling, Monitoring, and Control
stochastic CARE
stochastic coupled algebraic Riccatic equation
stochastic systems
Stochasticity
ST‐based data‐driven policy iteration algorithm
subsystems transformation technique
transforms
iterative methods
stochastic coupled algebraic Riccatic equation
multiplicative noises
subsystems transformation technique
parallel Kleinman iterative equations
transforms
optimal control
offline iteration algorithm
convergence of numerical methods
ST-based data-driven policy iteration algorithm
Riccati equations
infinite horizon optimal control problem
continuous-time Itô stochastic systems
stochastic systems
Markov processes
Markovian jumps
continuous time systems
implicit iterative algorithm
stochastic CARE
ISSN:1751-8644, 1751-8652
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This studies the infinite horizon optimal control problem for a class of continuous-time systems subjected to multiplicative noises and Markovian jumps by using a data-driven policy iteration algorithm. The optimal control problem is equivalent to solve a stochastic coupled algebraic Riccatic equation (CARE). An off-line iteration algorithm is first established to converge the solutions of the stochastic CARE, which is generalised from an implicit iterative algorithm. By applying subsystems transformation (ST) technique, the off-line iterative algorithm is decoupled into N parallel Kleinman's iterative equations. To learn the solution of the stochastic CARE from N decomposed linear subsystems data, an ST-based data-driven policy iteration algorithm is proposed and the convergence is proved. Finally, a numerical example is given to illustrate the effectiveness and applicability of the proposed two iterative algorithms.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:1751-8644
1751-8652
DOI:10.1049/iet-cta.2015.0973