Incremental Dual Heuristic Dynamic Programming Based Hybrid Approach for Multi-Channel Control of Unstable Tailless Aircraft

Actor-critic based online reinforcement learning control has been proved to be promising method for control of aerial vehicles. However, it is difficult to guarantee high-level success rate of initial training and to tune the large amount of parameters for actors and critics considering unstable mul...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	IEEE access Ročník 10; s. 31677 - 31691
Hlavní autori:	Li, Hangxu, Sun, Liguo, Tan, Wenqian, Liu, Xiaoyu, Dang, Weigao
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	Piscataway IEEE 2022 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:	actor and critic Adaptation models Adaptive control Aerospace control Aircraft Aircraft control Aircraft performance Aircraft stability Atmospheric modeling Augmentation systems Control methods Controllers Critics Dynamic programming Eigenvalues Heuristic Hybrid control Incremental DHP Mathematical models MIMO communication MIMO control Neural networks Process parameters reinforcement learning Robust control Stability augmentation Synthetic aperture sonar Tailless aircraft Training unstable aircraft
ISSN:	2169-3536, 2169-3536
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Abstract	Actor-critic based online reinforcement learning control has been proved to be promising method for control of aerial vehicles. However, it is difficult to guarantee high-level success rate of initial training and to tune the large amount of parameters for actors and critics considering unstable multi-input and multi-output (MIMO) aircraft. In order to facilitate and simplify the training of the actor and critic for unstable aircraft, classic stability augmentation system (SAS) is designed for the open-loop aircraft first. Then the online incremental model based dual heuristic dynamic programming (IDHP) method, which has been proposed recently, is extended in application to design a multi-channel robust adaptive controller, and MIMO form network structures are designed and determined for the actors and critics considering the three-channel coupling issues. Consequently, the classic SAS and the IDHP controller make up a novel hybrid control framework. In this control framework, the SAS takes charge of counteracting the unstable eigenvalues of the open-loop aircraft system, and the IDHP takes charge on guaranteeing robust and adaptive performance for high-performance tailless aircraft equipped with the SAS. Specifically, the introduction of the classic control method decreases the difficulty of the initial training for multi-channel IDHP controller. The tuning process for initial parameters of actor and critic neural networks in multiple channels is greatly facilitated. Without the help of SAS, the initial training for multi-channel IDHP controllers of unstable plants is almost impossible to succeed. Finally, the novel hybrid control architecture and method are validated using the Innovative Control Effectors (ICE) model, which has unstable modes in the longitudinal dynamics. Typical aerodynamic model uncertainties are numerically simulated to demonstrate the effectiveness of the proposed control method.
AbstractList	Actor-critic based online reinforcement learning control has been proved to be promising method for control of aerial vehicles. However, it is difficult to guarantee high-level success rate of initial training and to tune the large amount of parameters for actors and critics considering unstable multi-input and multi-output (MIMO) aircraft. In order to facilitate and simplify the training of the actor and critic for unstable aircraft, classic stability augmentation system (SAS) is designed for the open-loop aircraft first. Then the online incremental model based dual heuristic dynamic programming (IDHP) method, which has been proposed recently, is extended in application to design a multi-channel robust adaptive controller, and MIMO form network structures are designed and determined for the actors and critics considering the three-channel coupling issues. Consequently, the classic SAS and the IDHP controller make up a novel hybrid control framework. In this control framework, the SAS takes charge of counteracting the unstable eigenvalues of the open-loop aircraft system, and the IDHP takes charge on guaranteeing robust and adaptive performance for high-performance tailless aircraft equipped with the SAS. Specifically, the introduction of the classic control method decreases the difficulty of the initial training for multi-channel IDHP controller. The tuning process for initial parameters of actor and critic neural networks in multiple channels is greatly facilitated. Without the help of SAS, the initial training for multi-channel IDHP controllers of unstable plants is almost impossible to succeed. Finally, the novel hybrid control architecture and method are validated using the Innovative Control Effectors (ICE) model, which has unstable modes in the longitudinal dynamics. Typical aerodynamic model uncertainties are numerically simulated to demonstrate the effectiveness of the proposed control method.
Author	Li, Hangxu Sun, Liguo Liu, Xiaoyu Tan, Wenqian Dang, Weigao
Author_xml	– sequence: 1 givenname: Hangxu orcidid: 0000-0003-3321-2248 surname: Li fullname: Li, Hangxu organization: School of Aeronautic Science and Engineering, Beihang University, Beijing, China – sequence: 2 givenname: Liguo orcidid: 0000-0002-5552-8289 surname: Sun fullname: Sun, Liguo organization: School of Aeronautic Science and Engineering, Beihang University, Beijing, China – sequence: 3 givenname: Wenqian orcidid: 0000-0002-8147-5731 surname: Tan fullname: Tan, Wenqian email: tanwenqian@buaa.edu.cn organization: School of Aeronautic Science and Engineering, Beihang University, Beijing, China – sequence: 4 givenname: Xiaoyu orcidid: 0000-0002-5200-9563 surname: Liu fullname: Liu, Xiaoyu organization: School of Aeronautic Science and Engineering, Beihang University, Beijing, China – sequence: 5 givenname: Weigao surname: Dang fullname: Dang, Weigao organization: School of Aeronautic Science and Engineering, Beihang University, Beijing, China
BookMark	eNp9UU1r3DAQNSGFpEl-QS6Cnr3VlyXruHXS7kJKA0nOQpLljRZb2kryYaE_vto6LaWHzmFGesx7zMx7X5374G1V3SK4QgiKj-uuu396WmGI8Yqghre8PasuMWKiJg1h53-9L6qblPawRFughl9WP7beRDtZn9UI7uaSNnaOLmVnwN3Rq6nUxxh2UU2T8zvwSSXbg81RR9eD9eEQgzKvYAgRfJ3H7OruVXlvR9AFn2MYQRjAi09Z6dGCZ-XG0aYE1i6aqIZ8Xb0b1JjszVu9ql4-3z93m_rh25dtt36oDYVtrltGBwip0Aw1TDPKucG6UYpZjFtKyscO3GjKNWkY1HwQqsCsJxw3AglNrqrtotsHtZeH6CYVjzIoJ38BIe6kimXj0cq-pZoRLWzDB0pErzWllDFlWMst6mnR-rBoldW_zzZluQ9z9GV8iRklEDeIkNIlli4TQ0rRDtK4rLI7XaVcQSIoT97JxTt58k6-eVe45B_u74n_z7pdWM5a-4chOKGQYPITci2nOw
CODEN	IAECCG
CitedBy_id	crossref_primary_10_3390_aerospace10030241 crossref_primary_10_3390_drones7050294
Cites_doi	10.1016/j.ast.2020.106013 10.1109/72.914523 10.1109/TSG.2014.2346740 10.1016/j.ast.2020.106233 10.2514/1.G003472 10.2514/6.2006-6429 10.1016/j.ast.2020.106126 10.1016/j.cja.2020.02.020 10.2514/6.2020-1844 10.1016/j.ast.2020.105986 10.2514/6.2019-2339 10.1016/j.conengprac.2017.12.011 10.1109/MCI.2009.932261 10.2514/6.2017-1757 10.2514/6.2021-0884 10.1016/j.ins.2012.07.014 10.1109/CDC.2002.1184694 10.1109/TNN.2003.813839 10.1109/TIE.2017.2653767 10.1109/72.623201 10.2514/1.G004519 10.1109/TNN.2002.1000146 10.1109/ICARES.2019.8914343
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022
DBID	97E ESBDL RIA RIE AAYXX CITATION 7SC 7SP 7SR 8BQ 8FD JG9 JQ2 L7M L~C L~D DOA
DOI	10.1109/ACCESS.2022.3157878
DatabaseName	IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE/IET Electronic Library (IEL) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE/IET Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Engineered Materials Abstracts METADEX Technology Research Database Materials Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional DOAJ Directory of Open Access Journals
DatabaseTitle	CrossRef Materials Research Database Engineered Materials Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace METADEX Computer and Information Systems Abstracts Professional
DatabaseTitleList	Materials Research Database
Database_xml	– sequence: 1 dbid: DOA name: Directory of Open Access Journals url: https://www.doaj.org/ sourceTypes: Open Website – sequence: 2 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	2169-3536
EndPage	31691
ExternalDocumentID	oai_doaj_org_article_d84b63b9e57f439dbb44466ac687e1d4 10_1109_ACCESS_2022_3157878 9734032
Genre	orig-research
GrantInformation_xml	– fundername: National Natural Science Foundation of China grantid: 11502008 funderid: 10.13039/501100001809 – fundername: Fundamental Research Funds for the Central Universities of China grantid: YWF-21-BJ-J-809 funderid: 10.13039/501100012226 – fundername: Aeronautical Science Foundation of China grantid: 20185702003 funderid: 10.13039/501100004750
GroupedDBID	0R~ 4.4 5VS 6IK 97E AAJGR ABAZT ABVLG ACGFS ADBBV AGSQL ALMA_UNASSIGNED_HOLDINGS BCNDV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD ESBDL GROUPED_DOAJ IPLJI JAVBF KQ8 M43 M~E O9- OCL OK1 RIA RIE RNS AAYXX CITATION 7SC 7SP 7SR 8BQ 8FD JG9 JQ2 L7M L~C L~D
ID	FETCH-LOGICAL-c408t-864f0049b6156b6477c2b5aa6e22843c2bef7cb47b3560b7f9a8436d3725919b3
IEDL.DBID	DOA
ISICitedReferencesCount	3
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000773237500001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	2169-3536
IngestDate	Fri Oct 03 12:53:44 EDT 2025 Mon Jun 30 03:13:52 EDT 2025 Sat Nov 29 06:31:59 EST 2025 Tue Nov 18 21:49:03 EST 2025 Wed Aug 27 02:40:37 EDT 2025
IsDoiOpenAccess	true
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Language	English
License	https://creativecommons.org/licenses/by-nc-nd/4.0
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c408t-864f0049b6156b6477c2b5aa6e22843c2bef7cb47b3560b7f9a8436d3725919b3
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0002-5552-8289 0000-0003-3321-2248 0000-0002-8147-5731 0000-0002-5200-9563
OpenAccessLink	https://doaj.org/article/d84b63b9e57f439dbb44466ac687e1d4
PQID	2643025133
PQPubID	4845423
PageCount	15
ParticipantIDs	crossref_citationtrail_10_1109_ACCESS_2022_3157878 doaj_primary_oai_doaj_org_article_d84b63b9e57f439dbb44466ac687e1d4 crossref_primary_10_1109_ACCESS_2022_3157878 proquest_journals_2643025133 ieee_primary_9734032
PublicationCentury	2000
PublicationDate	20220000 2022-00-00 20220101 2022-01-01
PublicationDateYYYYMMDD	2022-01-01
PublicationDate_xml	– year: 2022 text: 20220000
PublicationDecade	2020
PublicationPlace	Piscataway
PublicationPlace_xml	– name: Piscataway
PublicationTitle	IEEE access
PublicationTitleAbbrev	Access
PublicationYear	2022
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 ref12 ref15 ref14 ref11 ref10 Kirk (ref20) 2012 ref2 ref1 ref17 ref16 ref19 ref18 ref26 ref25 ref22 ref21 ref8 ref7 ref9 Dorsett (ref23) 1996 Dorsett (ref24) 1997 ref4 ref3 ref6 ref5
References_xml	– ident: ref13 doi: 10.1016/j.ast.2020.106013 – ident: ref5 doi: 10.1109/72.914523 – ident: ref8 doi: 10.1109/TSG.2014.2346740 – ident: ref9 doi: 10.1016/j.ast.2020.106233 – ident: ref12 doi: 10.2514/1.G003472 – ident: ref21 doi: 10.2514/6.2006-6429 – ident: ref18 doi: 10.1016/j.ast.2020.106126 – ident: ref25 doi: 10.1016/j.cja.2020.02.020 – ident: ref16 doi: 10.2514/6.2020-1844 – ident: ref19 doi: 10.1016/j.ast.2020.105986 – ident: ref15 doi: 10.2514/6.2019-2339 – volume-title: Optimal Control Theory: An Introduction year: 2012 ident: ref20 – ident: ref6 doi: 10.1016/j.conengprac.2017.12.011 – ident: ref17 doi: 10.1109/MCI.2009.932261 – year: 1997 ident: ref24 article-title: Innovative control effectors (ICE) phase III – ident: ref22 doi: 10.2514/6.2017-1757 – ident: ref10 doi: 10.2514/6.2021-0884 – ident: ref2 doi: 10.1016/j.ins.2012.07.014 – ident: ref26 doi: 10.1109/CDC.2002.1184694 – ident: ref7 doi: 10.1109/TNN.2003.813839 – year: 1996 ident: ref23 article-title: Innovative control effectors (ICE) – ident: ref1 doi: 10.1109/TIE.2017.2653767 – ident: ref3 doi: 10.1109/72.623201 – ident: ref14 doi: 10.2514/1.G004519 – ident: ref4 doi: 10.1109/TNN.2002.1000146 – ident: ref11 doi: 10.1109/ICARES.2019.8914343
SSID	ssj0000816957
Score	2.249511
Snippet	Actor-critic based online reinforcement learning control has been proved to be promising method for control of aerial vehicles. However, it is difficult to...
SourceID	doaj proquest crossref ieee
SourceType	Open Website Aggregation Database Enrichment Source Index Database Publisher
StartPage	31677
SubjectTerms	actor and critic Adaptation models Adaptive control Aerospace control Aircraft Aircraft control Aircraft performance Aircraft stability Atmospheric modeling Augmentation systems Control methods Controllers Critics Dynamic programming Eigenvalues Heuristic Hybrid control Incremental DHP Mathematical models MIMO communication MIMO control Neural networks Process parameters reinforcement learning Robust control Stability augmentation Synthetic aperture sonar Tailless aircraft Training unstable aircraft
SummonAdditionalLinks	– databaseName: IEEE/IET Electronic Library (IEL) dbid: RIE link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PSxwxFA4qPeihtbXitlZy6NHR2Uk2mRzXtbKHIh4UvIX8eAPCdlf2hyD0j-97SRxaWoRehpmQDBm-ZN77krzvMfY1SCNdFHXllI-VNMpVBoSo2oC-RduJoY9JZ_a7vr5u7-_NzRY77WNhACAdPoMzuk17-XERNrRUdm60kLXAH-621irHavXrKZRAwox0ERYa1uZ8PJngNyAFbBpkpjQy2z-MT9LoL0lV_voTJ_Ny9e7_OrbP3hY3ko8z7u_ZFsw_sL3fxAUP2E-c-nnxDytebvAyhU3WZeaXOQ89v8mns35gA36B9izy6TOFcPFxkRrn6NPyFKRbURzCHGZ8kg-380XH78i39DPgt45CClcrPn5YhqXr1h_Z3dW328m0KrkWqiDrdl21SnbEFjx6OMpTdGpo_Mg5BQ0aMIEP0OngpfYCfSSvO-OwWEWhkT8NjReHbGe-mMMR41FLV7ca0CYGJH_S1IBsNxgHjdPG6wFrXkCwoQiRUz6MmU2EpDY2I2cJOVuQG7DTvtFj1uF4vfoFodtXJRHtVICw2TInbWylV8IbGOkO_bLovaTdbRcU9T7KATsgqPuXFJQH7PhlrNgy4VcW_UpBdE2IT_9u9ZntUgfz6s0x21kvN_CFvQlP64fV8iSN5V_WC_Gu priority: 102 providerName: IEEE
Title	Incremental Dual Heuristic Dynamic Programming Based Hybrid Approach for Multi-Channel Control of Unstable Tailless Aircraft
URI	https://ieeexplore.ieee.org/document/9734032 https://www.proquest.com/docview/2643025133 https://doaj.org/article/d84b63b9e57f439dbb44466ac687e1d4
Volume	10
WOSCitedRecordID	wos000773237500001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVAON databaseName: Directory of Open Access Journals customDbUrl: eissn: 2169-3536 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000816957 issn: 2169-3536 databaseCode: DOA dateStart: 20130101 isFulltext: true titleUrlDefault: https://www.doaj.org/ providerName: Directory of Open Access Journals – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2169-3536 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000816957 issn: 2169-3536 databaseCode: M~E dateStart: 20130101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV3NSxwxFA8iPeihtFVxWys59NjBmUk2H8fdVdmDXfag4C3ka0BY17IfBUH6t_e9JC4rBb14CcyQzEzyXvLeL5P3e4T88FxzG1hdWeFCxbWwlY6MVcqDb6E61riQeGav5GSibm_1dCvVF54Jy_TAeeDOguJOMKdjX3ZgPINzHH9BWi-UjE1ITKC11FtgKq3BqhG6LwvNUFPrs8FoBD0CQNi2gFNRT9ULU5QY-0uKlf_W5WRsLj-Rj8VLpIP8dZ_JTpx_Iftb3IEH5Almdt7bg4rnayjGcZ1pl-l5TjNPp_nw1T00oEMwV4GOHzFCiw4KkzgFl5WmGNwKwwzmcUZH-ew6fejoDbqObhbptcWIweWSDu4WfmG71SG5uby4Ho2rkkqh8rxWq0oJ3iEYcODACIfBp751fWtFbME-MbiInfSOS8fABXKy0xZui8AkwKNGO3ZEducP83hMaJDc1jj2LfOA7biuI4BZr21srdRO9kj7PKrGF55xTHcxMwlv1NpkURgUhSmi6JGfm0a_M83G69WHKK5NVeTITjdAc0zRHPOW5vTIAQp78xAtGa9Z2yMnz8I3ZT4vDbiNDNEYY1_f49XfyB52J2_lnJDd1WIdv5MP_s_qbrk4TaoM5a-_F6cpIPEfKRP3FA
linkProvider	Directory of Open Access Journals
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PaxQxFH6UWtAetFrF1Vpz8Nixs5PsZHLcbi1bXJcettBbyK-Bwror-6NQ6B_ve0k6KIrgZZgJyZDhS-a9L8n7HsAnJ5QwnpeFqa0vhKpNoQLnRePQt2ha3rc-6sxO5HTa3Nyoqx046WJhQgjx8Fn4TLdxL98v3ZaWyk6V5KLk-MN9MhCiKlO0VreiQikk1EBmaaF-qU6HoxF-BZLAqkJuSmOz-c38RJX-nFblj39xNDAXL_6vawfwPDuSbJiQfwk7YfEK9n-RFzyEB5z8afkPK55v8TIO26TMzM5TJnp2lc5nfccG7AwtmmfjewriYsMsNs7Qq2UxTLegSIRFmLNROt7Oli27Ju_SzgObGQoqXK_Z8HblVqbdvIbriy-z0bjI2RYKJ8pmUzS1aIkvWPRxakvxqa6yA2PqUKEJ4_gQWumskJajl2RlqwwW155LZFB9Zfkb2F0sF-EtMC-FKRsZ0Co6pH9ClQH5rlMmVEYqK3tQPYKgXZYip4wYcx0pSal0Qk4Tcjoj14OTrtGPpMTx7-pnhG5XlWS0YwHCpvOs1L4RtuZWhYFs0TPz1gra3zaupt570YNDgrp7SUa5B0ePY0XnKb_W6FlyImycv_t7q4_wdDz7NtGTy-nX9_CMOpvWco5gd7Pahg-w5-42t-vVcRzXPwGJkPT1
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Incremental+Dual+Heuristic+Dynamic+Programming+Based+Hybrid+Approach+for+Multi-Channel+Control+of+Unstable+Tailless+Aircraft&rft.jtitle=IEEE+access&rft.au=Li%2C+Hangxu&rft.au=Sun%2C+Liguo&rft.au=Tan%2C+Wenqian&rft.au=Liu%2C+Xiaoyu&rft.date=2022&rft.issn=2169-3536&rft.eissn=2169-3536&rft.volume=10&rft.spage=31677&rft.epage=31691&rft_id=info:doi/10.1109%2FACCESS.2022.3157878&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_ACCESS_2022_3157878
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2169-3536&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2169-3536&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2169-3536&client=summon