Neuro-Control for Continuous-Time Stochastic Nonlinear Systems via Online Policy Iteration Algorithm

This paper is concerned with the neuro-control for continuous-time nonlinear systems subject to stochastic disturbance. Due to the stochastic disturbance, the traditional value function in existing literature cannot meet the stochastic control problems, since mixed second partial derivatives are emp...

Full description

Saved in:
Bibliographic Details
Published in:Chinese Control and Decision Conference pp. 1499 - 1503
Main Authors: Zhou, Tianmin, Hou, Jiaxu, Li, Handong, Di, Zengru, Zhao, Bo
Format: Conference Proceeding
Language:English
Published: IEEE 01.08.2020
Subjects:
Adaptive Dynamic Programming
Approximation algorithms
Artificial neural networks
Mathematical model
Nonlinear systems
Optimal control
Policy Iteration
Reinforcement Learning
Stochastic Nonlinear
Stochastic processes
Stochastic systems
ISSN:1948-9447
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract This paper is concerned with the neuro-control for continuous-time nonlinear systems subject to stochastic disturbance. Due to the stochastic disturbance, the traditional value function in existing literature cannot meet the stochastic control problems, since mixed second partial derivatives are employed to construct modified value function of conditional expectation. To solve the Hamilton-Jacobi-Bellman equation, a novel online policy iteration algorithm with an Ito correction term is developed with establishing a critic neural network to approximate the optimal value function.ˆ Thus, the online optimal control can be obtained in a closed-loop form. The closed-loop system is guaranteed to be stable in probability via Lyapunov's direct method. Finally, numerical example is provided to illustrate the effectiveness of the developed control method.
AbstractList This paper is concerned with the neuro-control for continuous-time nonlinear systems subject to stochastic disturbance. Due to the stochastic disturbance, the traditional value function in existing literature cannot meet the stochastic control problems, since mixed second partial derivatives are employed to construct modified value function of conditional expectation. To solve the Hamilton-Jacobi-Bellman equation, a novel online policy iteration algorithm with an Ito correction term is developed with establishing a critic neural network to approximate the optimal value function.ˆ Thus, the online optimal control can be obtained in a closed-loop form. The closed-loop system is guaranteed to be stable in probability via Lyapunov's direct method. Finally, numerical example is provided to illustrate the effectiveness of the developed control method.
Author Zhou, Tianmin
Li, Handong
Hou, Jiaxu
Di, Zengru
Zhao, Bo
Author_xml – sequence: 1
  givenname: Tianmin
  surname: Zhou
  fullname: Zhou, Tianmin
  organization: Beijing Normal University,School of Systems Science,Beijing,China,100875
– sequence: 2
  givenname: Jiaxu
  surname: Hou
  fullname: Hou, Jiaxu
  organization: Beijing Normal University,School of Systems Science,Beijing,China,100875
– sequence: 3
  givenname: Handong
  surname: Li
  fullname: Li, Handong
  organization: Beijing Normal University,School of Systems Science,Beijing,China,100875
– sequence: 4
  givenname: Zengru
  surname: Di
  fullname: Di, Zengru
  organization: Beijing Normal University,School of Systems Science,Beijing,China,100875
– sequence: 5
  givenname: Bo
  surname: Zhao
  fullname: Zhao, Bo
  organization: Beijing Normal University,School of Systems Science,Beijing,China,100875
BookMark eNotkMtKAzEYRqMo2NY-gSB5gam5zCSTZRlvhdIKreuSSf6xkZlEkozQtxe1q-9wFmfxTdGVDx4QuqdkQSlRD03z2JSKM7VghJGFoqKUUl6guZI1laymVV1V4hJNqCrrQpWlvEHTlD4JEYITMkF2A2MMRRN8jqHHXYj4l50fw5iKvRsA73IwR52yM3gTfO886Ih3p5RhSPjbabz9k_gt9M6c8CpD1NkFj5f9R4guH4dbdN3pPsH8vDP0_vy0b16L9fZl1SzXhWOE54K1tbLWAu3KiktLjBEdSKpqQUQrFOmYAGmFZpKDaNtKM0uZMmCs4mVXdXyG7v67DgAOX9ENOp4O51P4D98tWrY
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1109/CCDC49329.2020.9164777
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Xplore POP ALL
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
EISBN 9781728158556
1728158540
1728158559
9781728158549
EISSN 1948-9447
EndPage 1503
ExternalDocumentID 9164777
Genre orig-research
GroupedDBID 29B
6IE
6IF
6IK
6IL
6IN
AAJGR
AAWTH
ABLEC
ADZIZ
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
IEGSK
IPLJI
M43
OCL
RIE
RIL
RNS
ID FETCH-LOGICAL-i203t-2b89ddde1f4537d0cc6fe7198606b690f26e7d6a273e6bb5a2d129cecd934f5f3
IEDL.DBID RIE
ISICitedReferencesCount 0
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000621616901105&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Aug 27 02:33:55 EDT 2025
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i203t-2b89ddde1f4537d0cc6fe7198606b690f26e7d6a273e6bb5a2d129cecd934f5f3
PageCount 5
ParticipantIDs ieee_primary_9164777
PublicationCentury 2000
PublicationDate 2020-Aug.
PublicationDateYYYYMMDD 2020-08-01
PublicationDate_xml – month: 08
  year: 2020
  text: 2020-Aug.
PublicationDecade 2020
PublicationTitle Chinese Control and Decision Conference
PublicationTitleAbbrev CCDC
PublicationYear 2020
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0066300
Score 1.7298181
Snippet This paper is concerned with the neuro-control for continuous-time nonlinear systems subject to stochastic disturbance. Due to the stochastic disturbance, the...
SourceID ieee
SourceType Publisher
StartPage 1499
SubjectTerms Adaptive Dynamic Programming
Approximation algorithms
Artificial neural networks
Mathematical model
Nonlinear systems
Optimal control
Policy Iteration
Reinforcement Learning
Stochastic Nonlinear
Stochastic processes
Stochastic systems
Title Neuro-Control for Continuous-Time Stochastic Nonlinear Systems via Online Policy Iteration Algorithm
URI https://ieeexplore.ieee.org/document/9164777
WOSCitedRecordID wos000621616901105&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlZ3LS8MwHMfDHB704mMT3-Tg0W5JH0lzlOrQyxiosNvI4xdX0FZqt7_fpK1TwYu3UAiB_OD3aH7fzw-hKy2YJsrPCKNekkPA-UGIdaCFYopaITnRzbAJPp2m87mY9dD1RgsDAE3zGYz8snnLN6Ve-V9lY-HhV5xvoS3OWavV-vK6zKOjOgUwJWKcZbdZ7HITr0UJyajb-WuEShNBJnv_O3sfDb-leHi2CTIHqAfFIdr9QREcINMANoKsbTrHLgvFfp0XK1fVB17jgR_rUi-lRzLjacvGkBXuYOV4nUvcEkdxSwnGDw1q2VkM37y-lFVeL9-G6Hly95TdB93whCAPSVQHoUqFcb6L2jiJuCFaMwucitRVLMqVxDZkwA2TLn0BplQiQ-NCvwZtRBTbxEZHqF-UBRwjDEoRyyGJNYM40kIm0lBNqeFcW2LSEzTw97V4b_kYi-6qTv_-fIZ2vEnaJrpz1K-rFVygbb2u84_qsjHqJ-6-phw
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlZ1bS8MwFMfDnIL64mUT7-bBR7ulbZo0j1IdG84ycMLeRm51BW2ldvv8Jm2dCr74FgohkAPn0pz_7wBwLRmRSNgZYa6V5CBt_KDG0pFMEOEmjFMkq2ETNI7D2YxNWuBmrYXRWlfNZ7pnl9Vbvsrl0v4q6zMLv6J0A2wGGHuoVmt9-V1i4VGNBthFrB9FdxE22YlVo3io1-z9NUSliiGDvf-dvg-632I8OFmHmQPQ0tkh2P3BEewAVSE2nKhuO4cmD4V2nWZLU9c7VuUBn8pcLriFMsO4pmPwAja4crhKOayZo7DmBMNRBVs2NoO3ry95kZaLty54HtxPo6HTjE9wUg_5peOJkCnjvdwEBz5VSEqSaOqy0NQswhTFiUc0VYSbBEYTIQLuKRP8pZaK-TgJEv8ItLM808cAaiFQQnWAJdHYl4wHXLnSdRWlMkEqPAEde1_z95qQMW-u6vTvz1dgezh9HM_Ho_jhDOxY89QtdeegXRZLfQG25KpMP4rLysCfvUCpYw
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Chinese+Control+and+Decision+Conference&rft.atitle=Neuro-Control+for+Continuous-Time+Stochastic+Nonlinear+Systems+via+Online+Policy+Iteration+Algorithm&rft.au=Zhou%2C+Tianmin&rft.au=Hou%2C+Jiaxu&rft.au=Li%2C+Handong&rft.au=Di%2C+Zengru&rft.date=2020-08-01&rft.pub=IEEE&rft.eissn=1948-9447&rft.spage=1499&rft.epage=1503&rft_id=info:doi/10.1109%2FCCDC49329.2020.9164777&rft.externalDocID=9164777