Fractional-Order Systems Optimal Control via Actor-Critic Reinforcement Learning and Its Validation for Chaotic MFET
Since the existence of fractional order dynamics, it is difficult to obtain an optimality equation to solve for fractional-order optimal control. In this paper, a fractional Hamilton-Jacobi-Bellman (HJB) equation based on error derivative is proposed, and a corresponding online learning algorithm is...
Saved in:
| Published in: | IEEE transactions on automation science and engineering Vol. 22; pp. 1173 - 1182 |
|---|---|
| Main Authors: | , |
| Format: | Journal Article |
| Language: | English |
| Published: |
IEEE
2025
|
| Subjects: | |
| ISSN: | 1545-5955, 1558-3783 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | Since the existence of fractional order dynamics, it is difficult to obtain an optimality equation to solve for fractional-order optimal control. In this paper, a fractional Hamilton-Jacobi-Bellman (HJB) equation based on error derivative is proposed, and a corresponding online learning algorithm is designed. The scheme can handle the optimal tracking problem for the <inline-formula> <tex-math notation="LaTeX">0< \alpha \leq 1 </tex-math></inline-formula> order nonlinear systems. Since the traditional quadratic cost function is unbounded at infinite time and the optimal control derived from the discounted cost function fails to stabilize the system asymptotically, a cost function based on the error derivative is proposed, which can avoid these problems, and the system is not restricted to be zero equilibrium. Then, the fractional HJB equation is derived by constructing an auxiliary signal without directly using the chain rule of differentiation. The optimality, stability and convergence of its solution are proved, and actor-critic neural networks (NNs) are established to perform the RL algorithm. Finally, the algorithm is applied to a chaotic magnetic-field electromechanical transducer (MFET) system to verify the effectiveness and advantages. Note to Practitioners-In engineering, fractional-order dynamics properties are used in many practical systems such as chaotic MFET systems, chaotic arch micro-electro-mechanical systems and fractional order resonant controllers, etc. For these fractional-order systems, traditional integer-order control methods cannot be applied. Moreover, optimal performance with minimal cost/resources can be achieved through optimal control. Therefore, a intelligent control method based on reinforcement learning is proposed in this paper to realize the optimal control for fractional-order systems. The proposed method is applicable to most fractional-order systems and can achieve stable control while optimizing the objective performance and reducing energy consumption. Finally, the proposed algorithm is successfully applied to the chaotic MFET systems. |
|---|---|
| AbstractList | Since the existence of fractional order dynamics, it is difficult to obtain an optimality equation to solve for fractional-order optimal control. In this paper, a fractional Hamilton-Jacobi-Bellman (HJB) equation based on error derivative is proposed, and a corresponding online learning algorithm is designed. The scheme can handle the optimal tracking problem for the <inline-formula> <tex-math notation="LaTeX">0< \alpha \leq 1 </tex-math></inline-formula> order nonlinear systems. Since the traditional quadratic cost function is unbounded at infinite time and the optimal control derived from the discounted cost function fails to stabilize the system asymptotically, a cost function based on the error derivative is proposed, which can avoid these problems, and the system is not restricted to be zero equilibrium. Then, the fractional HJB equation is derived by constructing an auxiliary signal without directly using the chain rule of differentiation. The optimality, stability and convergence of its solution are proved, and actor-critic neural networks (NNs) are established to perform the RL algorithm. Finally, the algorithm is applied to a chaotic magnetic-field electromechanical transducer (MFET) system to verify the effectiveness and advantages. Note to Practitioners-In engineering, fractional-order dynamics properties are used in many practical systems such as chaotic MFET systems, chaotic arch micro-electro-mechanical systems and fractional order resonant controllers, etc. For these fractional-order systems, traditional integer-order control methods cannot be applied. Moreover, optimal performance with minimal cost/resources can be achieved through optimal control. Therefore, a intelligent control method based on reinforcement learning is proposed in this paper to realize the optimal control for fractional-order systems. The proposed method is applicable to most fractional-order systems and can achieve stable control while optimizing the objective performance and reducing energy consumption. Finally, the proposed algorithm is successfully applied to the chaotic MFET systems. |
| Author | Li, Dongdong Dong, Jiuxiang |
| Author_xml | – sequence: 1 givenname: Dongdong orcidid: 0000-0002-8814-146X surname: Li fullname: Li, Dongdong email: lidongdongyq@163.com organization: College of Information Science and Engineering, the State Key Laboratory of Synthetical Automation for Process Industries, and the Key Laboratory of Vibration and Control of Aero-Propulsion Systems Ministry of Education, Northeastern University, Shenyang, China – sequence: 2 givenname: Jiuxiang orcidid: 0000-0002-0318-3004 surname: Dong fullname: Dong, Jiuxiang email: dongjiuxiang@ise.neu.edu.cn organization: College of Information Science and Engineering, the State Key Laboratory of Synthetical Automation for Process Industries, and the Key Laboratory of Vibration and Control of Aero-Propulsion Systems Ministry of Education, Northeastern University, Shenyang, China |
| BookMark | eNp9kMtKAzEUhoNUsFYfQHCRF5ia61yWZWi1UCnY6nY4zWQ0Mk1KEoS-vRnbhbhwk4ST_zuc812jkXVWI3RHyZRSUj1sZ5v5lBEmppznlFF-gcZUyjLjRclHw1vITFZSXqHrED5JSpYVGaO48KCicRb6bO1b7fHmGKLeB7w-RLOHHtfORu96_GUAz1R0Pqu9iUbhF21s57zSe20jXmnw1th3DLbFyxjwG_SmhaE1Tilcf4AbqOfFfHuDLjvog7493xP0mqr1U7ZaPy7r2SpTLM9jxjpgSrBKthxylfYodCuHoxV5CaB41zJSiGoHZEcq3mlS5jKnHZOCs1ITPkHFqa_yLgSvu0aZ-DNS9GD6hpJmkNcM8ppBXnOWl0j6hzz4ZMMf_2XuT4zRWv_KC0FI-v4GE-J9qA |
| CODEN | ITASC7 |
| CitedBy_id | crossref_primary_10_3390_fractalfract8050256 crossref_primary_10_1109_TCSII_2025_3589991 crossref_primary_10_1002_asjc_3751 crossref_primary_10_1002_asjc_3833 crossref_primary_10_1109_TASE_2024_3487878 crossref_primary_10_1109_TASE_2025_3548986 crossref_primary_10_1109_TASE_2025_3592290 crossref_primary_10_3390_math13132078 |
| Cites_doi | 10.1007/978-3-642-18101-6 10.1016/j.amc.2022.127759 10.1007/s11071-020-05518-5 10.1109/TNNLS.2021.3051030 10.1109/TFUZZ.2022.3227993 10.1016/S0022-0728(71)80115-8 10.1109/TCYB.2022.3208124 10.1109/TCYB.2021.3088994 10.1109/TASE.2013.2280974 10.1109/TNNLS.2018.2803726 10.1016/j.sysconle.2015.07.004 10.1109/TNNLS.2021.3123444 10.1109/TSMC.2019.2933359 10.1109/TASE.2020.2998773 10.1016/j.automatica.2014.05.011 10.1016/j.automatica.2014.10.103 10.1016/j.chaos.2005.11.020 10.1016/j.automatica.2016.05.003 10.1109/TCST.2011.2153203 10.1007/s00521-012-1249-y 10.1109/TSMC.2021.3130718 10.1016/j.automatica.2010.02.018 10.1109/TNNLS.2021.3130092 10.1109/TNNLS.2022.3146889 10.1016/j.ins.2023.119577 10.1109/TNNLS.2015.2441749 10.1109/TSMC.2023.3320653 10.1109/TSMC.2021.3120432 10.2514/3.20641 10.1007/s11071-020-05776-3 10.1109/8.489308 10.1016/j.amc.2006.08.163 10.1109/TASE.2020.2996160 10.1109/TCYB.2021.3123377 10.1109/TFUZZ.2023.3294928 10.1109/TSMC.2018.2861826 10.1109/TAC.1984.1103551 10.1142/3779 10.1016/j.automatica.2020.109011 10.1109/TCYB.2021.3056990 10.1016/j.automatica.2004.11.034 10.1016/j.mechrescom.2012.08.003 10.1109/TCYB.2020.2982168 |
| ContentType | Journal Article |
| DBID | 97E RIA RIE AAYXX CITATION |
| DOI | 10.1109/TASE.2024.3361213 |
| DatabaseName | IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1558-3783 |
| EndPage | 1182 |
| ExternalDocumentID | 10_1109_TASE_2024_3361213 10440013 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: National Natural Science Foundation of China grantid: 62273079; 61420106016 funderid: 10.13039/501100001809 – fundername: Research Fund of State Key Laboratory for Synthetical Automation of Process Industries in China grantid: 2013ZCX01 – fundername: 1912 project – fundername: Fundamental Research Funds for the Central Universities in China grantid: N2004002; N2104005; N182608004 |
| GroupedDBID | -~X 0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK AENEX AETIX AGQYO AGSQL AHBIQ AIBXA AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD F5P HZ~ H~9 IFIPE IPLJI JAVBF LAI M43 O9- OCL PQQKQ RIA RIE RNS AAYXX CITATION |
| ID | FETCH-LOGICAL-c266t-2fa2c4295d3a6c7837ed537edd468aac3fd20749ba0b093fe086561f254328e03 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 20 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001177108100001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1545-5955 |
| IngestDate | Tue Nov 18 20:57:52 EST 2025 Sat Nov 29 08:16:50 EST 2025 Wed Aug 27 01:50:08 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c266t-2fa2c4295d3a6c7837ed537edd468aac3fd20749ba0b093fe086561f254328e03 |
| ORCID | 0000-0002-0318-3004 0000-0002-8814-146X |
| PageCount | 10 |
| ParticipantIDs | ieee_primary_10440013 crossref_citationtrail_10_1109_TASE_2024_3361213 crossref_primary_10_1109_TASE_2024_3361213 |
| PublicationCentury | 2000 |
| PublicationDate | 2025-00-00 |
| PublicationDateYYYYMMDD | 2025-01-01 |
| PublicationDate_xml | – year: 2025 text: 2025-00-00 |
| PublicationDecade | 2020 |
| PublicationTitle | IEEE transactions on automation science and engineering |
| PublicationTitleAbbrev | TASE |
| PublicationYear | 2025 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| References | ref13 ref35 Podlubny (ref9) 1999 ref12 ref34 ref15 ref37 ref14 ref36 ref31 ref30 ref11 ref33 ref10 ref32 ref2 ref1 ref17 ref39 ref16 ref38 ref19 ref18 ref24 ref23 Athans (ref43) 2013 ref45 ref26 ref25 ref20 ref42 ref41 ref22 ref44 ref21 ref28 ref27 ref29 ref8 ref7 ref4 ref3 ref6 ref5 ref40 |
| References_xml | – ident: ref10 doi: 10.1007/978-3-642-18101-6 – ident: ref39 doi: 10.1016/j.amc.2022.127759 – ident: ref40 doi: 10.1007/s11071-020-05518-5 – ident: ref33 doi: 10.1109/TNNLS.2021.3051030 – ident: ref22 doi: 10.1109/TFUZZ.2022.3227993 – ident: ref6 doi: 10.1016/S0022-0728(71)80115-8 – ident: ref19 doi: 10.1109/TCYB.2022.3208124 – ident: ref13 doi: 10.1109/TCYB.2021.3088994 – ident: ref31 doi: 10.1109/TASE.2013.2280974 – ident: ref21 doi: 10.1109/TNNLS.2018.2803726 – ident: ref8 doi: 10.1016/j.sysconle.2015.07.004 – ident: ref30 doi: 10.1109/TNNLS.2021.3123444 – ident: ref41 doi: 10.1109/TSMC.2019.2933359 – ident: ref7 doi: 10.1109/TASE.2020.2998773 – ident: ref27 doi: 10.1016/j.automatica.2014.05.011 – ident: ref42 doi: 10.1016/j.automatica.2014.10.103 – ident: ref1 doi: 10.1016/j.chaos.2005.11.020 – ident: ref28 doi: 10.1016/j.automatica.2016.05.003 – ident: ref37 doi: 10.1109/TCST.2011.2153203 – ident: ref25 doi: 10.1007/s00521-012-1249-y – ident: ref18 doi: 10.1109/TSMC.2021.3130718 – ident: ref26 doi: 10.1016/j.automatica.2010.02.018 – ident: ref35 doi: 10.1109/TNNLS.2021.3130092 – ident: ref17 doi: 10.1109/TNNLS.2022.3146889 – ident: ref23 doi: 10.1016/j.ins.2023.119577 – ident: ref34 doi: 10.1109/TNNLS.2015.2441749 – ident: ref44 doi: 10.1109/TSMC.2023.3320653 – ident: ref38 doi: 10.1109/TSMC.2021.3120432 – ident: ref3 doi: 10.2514/3.20641 – ident: ref12 doi: 10.1007/s11071-020-05776-3 – ident: ref4 doi: 10.1109/8.489308 – ident: ref11 doi: 10.1016/j.amc.2006.08.163 – ident: ref29 doi: 10.1109/TASE.2020.2996160 – ident: ref14 doi: 10.1109/TCYB.2021.3123377 – ident: ref20 doi: 10.1109/TFUZZ.2023.3294928 – ident: ref36 doi: 10.1109/TSMC.2018.2861826 – ident: ref5 doi: 10.1109/TAC.1984.1103551 – ident: ref2 doi: 10.1142/3779 – volume-title: Fractional Differential Equations year: 1999 ident: ref9 – ident: ref16 doi: 10.1016/j.automatica.2020.109011 – ident: ref15 doi: 10.1109/TCYB.2021.3056990 – ident: ref24 doi: 10.1016/j.automatica.2004.11.034 – ident: ref45 doi: 10.1016/j.mechrescom.2012.08.003 – ident: ref32 doi: 10.1109/TCYB.2020.2982168 – volume-title: Optimal Control: An Introduction to the Theory year: 2013 ident: ref43 |
| SSID | ssj0024890 |
| Score | 2.4878926 |
| Snippet | Since the existence of fractional order dynamics, it is difficult to obtain an optimality equation to solve for fractional-order optimal control. In this... |
| SourceID | crossref ieee |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 1173 |
| SubjectTerms | adaptive dynamic programming Automation Cost function Costs Fractional-order systems Heuristic algorithms neural networks Nonlinear systems Optimal control reinforcement learning (RL) Trajectory |
| Title | Fractional-Order Systems Optimal Control via Actor-Critic Reinforcement Learning and Its Validation for Chaotic MFET |
| URI | https://ieeexplore.ieee.org/document/10440013 |
| Volume | 22 |
| WOSCitedRecordID | wos001177108100001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-3783 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0024890 issn: 1545-5955 databaseCode: RIE dateStart: 20040101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA62eNCDz4r1RQ6ehNR0k33kWEqLgraitfS2ZPPQQm2l3fb3m5daDwpewrLMwLKT3flmJvMNAJeU2_4PTBDjSiIqtESZNEBOciykZLJQrlwwvEt7vWw0Yg-hWd31wiil3OEz1bCXrpYvZ2JpU2XmC6fUYpYKqKRp4pu1von1MpdQsZAAxSyOQwmzidn1oPXUMaFgRBuEWMYs8sMJrU1VcU6lu_vPx9kDOwE9wpY39z7YUNMDsL3GKXgIyu7c9yrwCepbXk0YSMlh3_wd3ox2259Oh6sxhy2bs0d-3gF8VI5GVbiMIQzMqy-QTyW8LRdwaCC7n8AEjRRsv_KZ1brvdgY18GzW9g0KoxWQMB65RJHmkTCuKJaEJyI1UaqSsV0kTTLOBdEyMuCCFRwXmBGtTORjkJa2rfNRpjA5AtXpbKqOARRaC0WSRBksRKVuFrGOU1E0M4lpkemoDvDnu85F4B234y8muYs_MMuteXJrnjyYpw6uvlTePenGX8I1a5o1QW-Vk1_un4KtyM7wdWmUM1At50t1DjbFqhwv5hduT30AXc_KQw |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LTwIxEG4UTdSDT4z47MGTSbFs22V7JAQikYdRJNw23T6UBMHAwu-37a6KB028NJvNTLLZ6e58M9P5BoBrKlz_ByaIC60QlUahSFkgpwSWSnGVaF8uGLSr3W40HPKHvFnd98Jorf3hM112l76Wr6Zy4VJl9gun1GGWdbDBKA1w1q71Ta0X-ZSKAwWIccbyImYF89t-7alhg8GAlglxnFnkhxtamavi3Upz758PtA92c_wIa5nBD8CanhyCnRVWwSOQNmdZt4IYo55j1oQ5LTns2f_Dm9WuZ-fT4XIkYM1l7VE28QA-ak-kKn3OEObcqy9QTBRspXM4sKA9m8EErRSsv4qp0-o0G_0ieLZr_Q7lwxWQtD45RYERgbTOiCkiQlm1capWzC2KhpEQkhgVWHjBE4ETzInRNvaxWMu45vkg0pgcg8JkOtEnAEpjpCZhqC0aospUEmZYVSaVSGGaRCYoAfz5rmOZM4-7ARjj2EcgmMfOPLEzT5ybpwRuvlTeM9qNv4SLzjQrgplVTn-5fwW27vqddtxude_PwHbgJvr6pMo5KKSzhb4Am3KZjuazS7-_PgAOGs2K |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Fractional-Order+Systems+Optimal+Control+via+Actor-Critic+Reinforcement+Learning+and+Its+Validation+for+Chaotic+MFET&rft.jtitle=IEEE+transactions+on+automation+science+and+engineering&rft.au=Li%2C+Dongdong&rft.au=Dong%2C+Jiuxiang&rft.date=2025&rft.issn=1545-5955&rft.eissn=1558-3783&rft.volume=22&rft.spage=1173&rft.epage=1182&rft_id=info:doi/10.1109%2FTASE.2024.3361213&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TASE_2024_3361213 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1545-5955&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1545-5955&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1545-5955&client=summon |