Adaptive Linear Quadratic Control for Stochastic Discrete-Time Linear Systems With Unmeasurable Multiplicative and Additive Noises

This note investigates the adaptive linear quadratic control problem (ALQCP) for stochastic discrete-time (DT) linear systems with unmeasurable multiplicative and additive noises. A data-driven value iteration algorithm is developed to solve the stochastic algebraic Riccati equation (SARE) that resu...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on automatic control Vol. 69; no. 11; pp. 7808 - 7815
Main Authors: Jiang, Yi, Liu, Lu, Feng, Gang
Format: Journal Article
Language:English
Published: New York IEEE 01.11.2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Adaptive control
Adaptive linear quadratic control
Adaptive systems
Additive noise
Classification algorithms
Discrete time systems
Heuristic algorithms
Iterative algorithms
Linear systems
Riccati equation
stochastic algebraic Riccati equation (SARE)
Stochastic processes
Symmetric matrices
System dynamics
unmeasurable noises
value iteration (VI)
ISSN:0018-9286, 1558-2523
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract This note investigates the adaptive linear quadratic control problem (ALQCP) for stochastic discrete-time (DT) linear systems with unmeasurable multiplicative and additive noises. A data-driven value iteration algorithm is developed to solve the stochastic algebraic Riccati equation (SARE) that results from the concerned problem and to simultaneously obtain the optimal feedback policy. The proposed algorithm directly uses online data to solve the ALQCP based on an unbiased estimator and an initial stabilizing controller with unknown system dynamics and unmeasurable multiplicative and additive noises. It is shown that the proposed algorithm under a finite length of online data converges to a neighborhood of the solution to the SARE with a probability and the input-to-state stability, and the neighborhood can be arbitrarily small while the probability can be arbitrarily close to one as the length of online data increases. The simulation results demonstrate the efficacy of the proposed algorithm.
AbstractList This note investigates the adaptive linear quadratic control problem (ALQCP) for stochastic discrete-time (DT) linear systems with unmeasurable multiplicative and additive noises. A data-driven value iteration algorithm is developed to solve the stochastic algebraic Riccati equation (SARE) that results from the concerned problem and to simultaneously obtain the optimal feedback policy. The proposed algorithm directly uses online data to solve the ALQCP based on an unbiased estimator and an initial stabilizing controller with unknown system dynamics and unmeasurable multiplicative and additive noises. It is shown that the proposed algorithm under a finite length of online data converges to a neighborhood of the solution to the SARE with a probability and the input-to-state stability, and the neighborhood can be arbitrarily small while the probability can be arbitrarily close to one as the length of online data increases. The simulation results demonstrate the efficacy of the proposed algorithm.
Author Liu, Lu
Feng, Gang
Jiang, Yi
Author_xml – sequence: 1
  givenname: Yi
  orcidid: 0000-0001-8927-0119
  surname: Jiang
  fullname: Jiang, Yi
  email: yjian22@cityu.edu.hk
  organization: Department of Biomedical Engineering, City University of Hong Kong, Hong Kong, China
– sequence: 2
  givenname: Lu
  orcidid: 0000-0003-2741-2542
  surname: Liu
  fullname: Liu, Lu
  email: luliu45@cityu.edu.hk
  organization: Department of Biomedical Engineering, City University of Hong Kong, Hong Kong, China
– sequence: 3
  givenname: Gang
  orcidid: 0000-0001-8508-8416
  surname: Feng
  fullname: Feng, Gang
  email: megfeng@cityu.edu.hk
  organization: Department of Biomedical Engineering, City University of Hong Kong, Hong Kong, China
BookMark eNp9kU1LAzEQhoNUsFXvHjwEPG_NxyabPZb6CVURKx6XNJmlKdtNTbKCV3-5WysiHjxlJsyTlzwzQoPWt4DQCSVjSkl5Pp9Mx4ywfMx5WUpO9tCQCqEyJhgfoCEhVGUlU_IAjWJc9a3MczpEHxOrN8m9AZ65FnTAj522QSdn8NS3KfgG1z7gp-TNUsft9YWLJkCCbO7WP9TTe0ywjvjFpSV-btegYxf0ogF81zXJbRpn9FeKbi2eWOu-mnvvIsQjtF_rJsLx93mInq8u59ObbPZwfTudzDLDSpYytiBFYZU15YLXQvPCWgYGBJGyECCF6ktutCW0XiiVayY0UcYWNc0JL4Xih-hs9-4m-NcOYqpWvgttH1lxyogUouCsnyK7KRN8jAHqahPcWof3ipJqa7rqTVdb09W36R6RfxDjUv_drT_tmv_A0x3oAOBXjuj3JCn_BD9Hjts
CODEN IETAA9
CitedBy_id crossref_primary_10_1016_j_automatica_2025_112121
crossref_primary_10_1109_TNNLS_2025_3558738
crossref_primary_10_1080_00207721_2025_2474137
Cites_doi 10.1016/j.automatica.2008.08.017
10.1007/978-3-540-68829-7
10.1109/TCYB.2018.2890046
10.1016/j.automatica.2022.110685
10.1016/j.automatica.2012.06.096
10.1109/TSMC.2020.3042876
10.1109/9.135491
10.1016/S0024-3795(00)00144-0
10.1137/21M1438797
10.1016/j.automatica.2016.05.003
10.1561/2600000023
10.1016/j.automatica.2022.110768
10.1016/j.automatica.2022.110486
10.1137/18M1214147
10.1109/TAC.2022.3172250
10.1007/978-3-030-60990-0
10.1109/TAC.2020.3037046
10.1109/TAC.2004.834121
10.1137/0313076
10.1016/j.conengprac.2021.105042
10.1109/TAC.2021.3085510
10.1016/j.neucom.2018.04.018
ContentType Journal Article
Copyright Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
Copyright_xml – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
DBID 97E
RIA
RIE
AAYXX
CITATION
7SC
7SP
7TB
8FD
FR3
JQ2
L7M
L~C
L~D
DOI 10.1109/TAC.2024.3399630
DatabaseName IEEE Xplore (IEEE)
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE Electronic Library (IEL)
CrossRef
Computer and Information Systems Abstracts
Electronics & Communications Abstracts
Mechanical & Transportation Engineering Abstracts
Technology Research Database
Engineering Research Database
ProQuest Computer Science Collection
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts – Academic
Computer and Information Systems Abstracts Professional
DatabaseTitle CrossRef
Technology Research Database
Computer and Information Systems Abstracts – Academic
Mechanical & Transportation Engineering Abstracts
Electronics & Communications Abstracts
ProQuest Computer Science Collection
Computer and Information Systems Abstracts
Engineering Research Database
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts Professional
DatabaseTitleList
Technology Research Database
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
EISSN 1558-2523
EndPage 7815
ExternalDocumentID 10_1109_TAC_2024_3399630
10528661
Genre orig-research
GrantInformation_xml – fundername: Research Grants Council of Hong Kong
  grantid: 11205221; 11210222
– fundername: NSFC-Excellent Young Scientists Fund (Hong Kong and Macao)
  grantid: 62222318
– fundername: Research Grants Council of Hong Kong
  grantid: PDFS2324-1S02
GroupedDBID -~X
.DC
0R~
29I
3EH
4.4
5GY
5VS
6IK
97E
AAJGR
AARMG
AASAJ
AAWTH
ABAZT
ABQJQ
ABVLG
ACGFO
ACGFS
ACIWK
ACNCT
AENEX
AETIX
AGQYO
AGSQL
AHBIQ
AI.
AIBXA
AKJIK
AKQYR
ALLEH
ALMA_UNASSIGNED_HOLDINGS
ASUFR
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CS3
DU5
EBS
EJD
F5P
HZ~
H~9
IAAWW
IBMZZ
ICLAB
IDIHD
IFIPE
IFJZH
IPLJI
JAVBF
LAI
M43
MS~
O9-
OCL
P2P
RIA
RIE
RNS
TAE
TN5
VH1
VJK
~02
AAYXX
CITATION
7SC
7SP
7TB
8FD
FR3
JQ2
L7M
L~C
L~D
ID FETCH-LOGICAL-c292t-2b077d8dc9b3f5a37dd2ece506675e658e503cad01fb884a25a08cd7f14039583
IEDL.DBID RIE
ISICitedReferencesCount 7
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001342803200025&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 0018-9286
IngestDate Mon Jun 30 10:18:36 EDT 2025
Sat Nov 29 05:41:12 EST 2025
Tue Nov 18 22:13:31 EST 2025
Wed Aug 27 02:14:36 EDT 2025
IsPeerReviewed true
IsScholarly true
Issue 11
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
https://doi.org/10.15223/policy-029
https://doi.org/10.15223/policy-037
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c292t-2b077d8dc9b3f5a37dd2ece506675e658e503cad01fb884a25a08cd7f14039583
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0000-0001-8927-0119
0000-0003-2741-2542
0000-0001-8508-8416
PQID 3120655732
PQPubID 85475
PageCount 8
ParticipantIDs proquest_journals_3120655732
ieee_primary_10528661
crossref_citationtrail_10_1109_TAC_2024_3399630
crossref_primary_10_1109_TAC_2024_3399630
PublicationCentury 2000
PublicationDate 2024-11-01
PublicationDateYYYYMMDD 2024-11-01
PublicationDate_xml – month: 11
  year: 2024
  text: 2024-11-01
  day: 01
PublicationDecade 2020
PublicationPlace New York
PublicationPlace_xml – name: New York
PublicationTitle IEEE transactions on automatic control
PublicationTitleAbbrev TAC
PublicationYear 2024
Publisher IEEE
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml – name: IEEE
– name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References ref13
ref12
ref15
ref14
ref11
ref10
ref2
ref1
ref16
ref19
ref18
Krauth (ref17) 2019
ref24
ref20
ref22
ref21
ref8
ref7
Astrm (ref23) 2013
ref9
ref4
ref3
ref6
ref5
Doob (ref25) 1953
References_xml – ident: ref4
  doi: 10.1016/j.automatica.2008.08.017
– ident: ref24
  doi: 10.1007/978-3-540-68829-7
– ident: ref7
  doi: 10.1109/TCYB.2018.2890046
– ident: ref18
  doi: 10.1016/j.automatica.2022.110685
– ident: ref5
  doi: 10.1016/j.automatica.2012.06.096
– start-page: 8514
  volume-title: Proc. Adv. Neural Inf. Process. Syst.
  year: 2019
  ident: ref17
  article-title: Finite-time analysis of approximate policy iteration for the linear quadratic regulator
– ident: ref2
  doi: 10.1109/TSMC.2020.3042876
– ident: ref10
  doi: 10.1109/9.135491
– ident: ref9
  doi: 10.1016/S0024-3795(00)00144-0
– ident: ref12
  doi: 10.1137/21M1438797
– volume-title: Adaptive Control
  year: 2013
  ident: ref23
– ident: ref6
  doi: 10.1016/j.automatica.2016.05.003
– ident: ref1
  doi: 10.1561/2600000023
– ident: ref22
  doi: 10.1016/j.automatica.2022.110768
– ident: ref8
  doi: 10.1016/j.automatica.2022.110486
– ident: ref11
  doi: 10.1137/18M1214147
– ident: ref14
  doi: 10.1109/TAC.2022.3172250
– volume-title: Stochastic Processes
  year: 1953
  ident: ref25
– ident: ref3
  doi: 10.1007/978-3-030-60990-0
– ident: ref16
  doi: 10.1109/TAC.2020.3037046
– ident: ref20
  doi: 10.1109/TAC.2004.834121
– ident: ref19
  doi: 10.1137/0313076
– ident: ref21
  doi: 10.1016/j.conengprac.2021.105042
– ident: ref13
  doi: 10.1109/TAC.2021.3085510
– ident: ref15
  doi: 10.1016/j.neucom.2018.04.018
SSID ssj0016441
Score 2.504779
Snippet This note investigates the adaptive linear quadratic control problem (ALQCP) for stochastic discrete-time (DT) linear systems with unmeasurable multiplicative...
SourceID proquest
crossref
ieee
SourceType Aggregation Database
Enrichment Source
Index Database
Publisher
StartPage 7808
SubjectTerms Adaptive control
Adaptive linear quadratic control
Adaptive systems
Additive noise
Classification algorithms
Discrete time systems
Heuristic algorithms
Iterative algorithms
Linear systems
Riccati equation
stochastic algebraic Riccati equation (SARE)
Stochastic processes
Symmetric matrices
System dynamics
unmeasurable noises
value iteration (VI)
Title Adaptive Linear Quadratic Control for Stochastic Discrete-Time Linear Systems With Unmeasurable Multiplicative and Additive Noises
URI https://ieeexplore.ieee.org/document/10528661
https://www.proquest.com/docview/3120655732
Volume 69
WOSCitedRecordID wos001342803200025&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVIEE
  databaseName: IEEE Electronic Library (IEL)
  customDbUrl:
  eissn: 1558-2523
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0016441
  issn: 0018-9286
  databaseCode: RIE
  dateStart: 19630101
  isFulltext: true
  titleUrlDefault: https://ieeexplore.ieee.org/
  providerName: IEEE
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELagYoCBN6JQkAcWhrTBjmtnjAqIhQpEK7pFju2ISpBWbcoP4Jdz56RVJQQSmwfbifTZ9_DdfUfIFRJ-ZKE1AarrIHIw0pKBs2Lz2OUi45Zr32xC9vtqNIqf6mJ1XwvjnPPJZ66NQx_LtxOzwKcyuOGCqS46O5tSdqtirVXIABV7JXbhBsO0ZUwyjDuDpAeeIIvaHNRxFxOe13SQb6ryQxJ79XK_988f2ye7tR1Jkwr4A7LhikOys8YueES-EqunKM0oOJxwoOnzQlsE3NBelaBOwWKlL-XEvGmka6a3YxAiYEUHWBiyXFVzmtPXcflGh8WHf1TEgiv6WCUj-lc_-IouLE2s9clItD8Zz938mAzv7wa9h6BuuRAYFrMyYFkopVXWxBnPhebSWuaME5gKKxxYKzDkRtvwJs-UijQTOlTGyhxp_2Kh-AlpFJPCnRLKsZU1-LzaRDpSocqwhBaDjNIKHTveJJ0lCKmp-cixLcZ76v2SME4BthRhS2vYmuR6tWJacXH8MfcYYVqbVyHUJK0l0Gl9W-cpv2FgiQnJ2dkvy87JNu5eFSG2SKOcLdwF2TKf5Xg-u_QH8RtcSNut
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA6ignrwWbE-c_DiYdttsulmj6VaFG1RbNHbkk2yWNCtdLf-AH-5M9mtFETBWw4JCXzJPDLzzRByjgU_Et9oD9W1F1gYqZCBs2LSyKYi4YYr12wiHAzk83N0X5HVHRfGWuuSz2wDhy6WbyZ6hl9l8MIFk210dlZEEDC_pGt9Bw1QtZeCF94wTJxHJf2oOex0wRdkQYODQm5jyvOCFnJtVX7IYqdgelv_PNo22awsSdopod8hSzbbJRsL9QX3yGfHqHeUZxRcTrjS9GGmDEKuabdMUadgs9LHYqJfFBZsppdjECNgR3tIDZmvqqqa06dx8UJH2Zv7VkTKFe2X6Yju3w92UZmhHWNcOhIdTMa5zWtk1Lsadq-9qumCp1nECo8lfhgaaXSU8FQoHhrDrLYCk2GFBXsFhlwr47fSRMpAMaF8qU2YYuG_SEi-T5azSWYPCOXYzBq8XqUDFUhfJkiixTBjaISKLK-T5hyEWFcVybExxmvsPBM_igG2GGGLK9jq5OJ7xXtZjeOPuTWEaWFeiVCdHM-Bjqv3mse8xcAWEyFnh78sOyNr18P-XXx3M7g9Iuu4U0lJPCbLxXRmT8iq_ijG-fTUXcov8Dje9A
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Adaptive+Linear+Quadratic+Control+for+Stochastic+Discrete-Time+Linear+Systems+With+Unmeasurable+Multiplicative+and+Additive+Noises&rft.jtitle=IEEE+transactions+on+automatic+control&rft.au=Jiang%2C+Yi&rft.au=Liu%2C+Lu&rft.au=Feng%2C+Gang&rft.date=2024-11-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.issn=0018-9286&rft.eissn=1558-2523&rft.volume=69&rft.issue=11&rft.spage=7808&rft_id=info:doi/10.1109%2FTAC.2024.3399630&rft.externalDBID=NO_FULL_TEXT
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9286&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9286&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9286&client=summon