A Novel Integral Reinforcement Learning-Based Control Method Assisted by Twin Delayed Deep Deterministic Policy Gradient for Solid Oxide Fuel Cell in DC Microgrid
This paper proposes a new online integral reinforcement learning (IRL)-based control algorithm for the solid oxide fuel cell (SOFC) to overcome the long-lasting problems of model dependency and sensitivity to offline training dataset in the existing SOFC control approaches. The proposed method autom...
Uloženo v:
| Vydáno v: | IEEE transactions on sustainable energy Ročník 14; číslo 1; s. 1 - 16 |
|---|---|
| Hlavní autoři: | , , , , , , , , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
Piscataway
IEEE
01.01.2023
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Témata: | |
| ISSN: | 1949-3029, 1949-3037 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | This paper proposes a new online integral reinforcement learning (IRL)-based control algorithm for the solid oxide fuel cell (SOFC) to overcome the long-lasting problems of model dependency and sensitivity to offline training dataset in the existing SOFC control approaches. The proposed method automatically updates the optimal control gains through the online neural network training. Unlike the other online learning-based control methods that rely on the assumption of initial stabilizing control or trial-and-error based initial control policy search, the proposed method employs the offline twin delayed deep deterministic policy gradient (TD3) algorithm to systematically determine the initial stabilizing control policy. Compared to the conventional IRL-based control, the proposed method contributes to greatly reduce the computational burden without compromising the control performance. The excellent performance of the proposed method is verified by hardware-in-the-loop experiments. |
|---|---|
| AbstractList | This paper proposes a new online integral reinforcement learning (IRL)-based control algorithm for the solid oxide fuel cell (SOFC) to overcome the long-lasting problems of model dependency and sensitivity to offline training dataset in the existing SOFC control approaches. The proposed method automatically updates the optimal control gains through the online neural network training. Unlike the other online learning-based control methods that rely on the assumption of initial stabilizing control or trial-and-error based initial control policy search, the proposed method employs the offline twin delayed deep deterministic policy gradient (TD3) algorithm to systematically determine the initial stabilizing control policy. Compared to the conventional IRL-based control, the proposed method contributes to greatly reduce the computational burden without compromising the control performance. The excellent performance of the proposed method is verified by hardware-in-the-loop experiments. |
| Author | Yu, Yang Zhang, Xinan Chau, Tat Kei Qie, Tianhao Iu, Herbert Fernando, Tyrone Manandhar, Ujjal Li, Sinan Liu, Yulin Wang, Yuxuan |
| Author_xml | – sequence: 1 givenname: Yulin orcidid: 0000-0001-7291-2242 surname: Liu fullname: Liu, Yulin organization: School of Engineering, University of Western Australia, Crawley, WA, Australia – sequence: 2 givenname: Tianhao orcidid: 0000-0002-4463-5927 surname: Qie fullname: Qie, Tianhao organization: School of Engineering, University of Western Australia, Crawley, WA, Australia – sequence: 3 givenname: Yang orcidid: 0000-0002-3694-3276 surname: Yu fullname: Yu, Yang organization: Center of excellence in advanced control, Halliburton Ltd, Singapore – sequence: 4 givenname: Yuxuan surname: Wang fullname: Wang, Yuxuan organization: School of Engineering, University of Western Australia, Crawley, WA, Australia – sequence: 5 givenname: Tat Kei orcidid: 0000-0001-9270-677X surname: Chau fullname: Chau, Tat Kei organization: School of Engineering, University of Western Australia, Crawley, WA, Australia – sequence: 6 givenname: Xinan orcidid: 0000-0002-9472-8785 surname: Zhang fullname: Zhang, Xinan organization: School of Engineering, University of Western Australia, Crawley, WA, Australia – sequence: 7 givenname: Ujjal orcidid: 0000-0003-2669-2095 surname: Manandhar fullname: Manandhar, Ujjal organization: School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore – sequence: 8 givenname: Sinan orcidid: 0000-0001-9519-2321 surname: Li fullname: Li, Sinan organization: School of Electrical and Information Engineering, The University of Sydney, Australia – sequence: 9 givenname: Herbert orcidid: 0000-0002-0687-4038 surname: Iu fullname: Iu, Herbert organization: School of Engineering, University of Western Australia, Crawley, WA, Australia – sequence: 10 givenname: Tyrone orcidid: 0000-0003-0140-8887 surname: Fernando fullname: Fernando, Tyrone organization: School of Engineering, University of Western Australia, Crawley, WA, Australia |
| BookMark | eNp9UctOGzEUtSoqQYEPQN1Y6nqCHxnPeJkOTylABWE9cuw7qZFjp7bTkt_hS-tREIsu6sW1fXUeV_d8QQc-eEDojJIJpUSeL54WlxNGGJtwxqa0kZ_QEZVTWXHCm4OPN5OH6DSlF1IO51xwcoTeZvg-_AaHb32GVVQOP4L1Q4ga1uAznoOK3vpV9V0lMLgLPsfg8B3kn8HgWUo25dJf7vDij_X4Apzalf8FwKaUDHFtfYFYjX8EZ_UOX0dl7KhcPPBT6Rn88GoN4KttmaID5_Co0-E7q2NYRWtO0OdBuQSn7_cxer66XHQ31fzh-rabzSvNJM8Vn9KlGrhoQQjCKOdaECokaxoxMMMUV1K2sjaC0laDqvWyUe3U1EtSD1oqyo_Rt73uJoZfW0i5fwnb6Itlz5pa1HVb07ag6B5VpkspwtBvol2ruOsp6cc0-jGNfkyjf0-jcJp_ONpmle24TGXdf5lf90wLAB9OUooxUv4XRSOZ0Q |
| CODEN | ITSEAJ |
| CitedBy_id | crossref_primary_10_1109_TTE_2024_3470240 crossref_primary_10_1109_TASE_2023_3309983 crossref_primary_10_1051_rees_2024001 crossref_primary_10_1109_TITS_2024_3462893 crossref_primary_10_1016_j_rser_2025_116000 crossref_primary_10_1109_TII_2024_3514084 crossref_primary_10_1109_TSG_2025_3567616 crossref_primary_10_1007_s42452_025_07529_6 crossref_primary_10_1016_j_ijepes_2024_110142 crossref_primary_10_1109_TSG_2023_3273239 crossref_primary_10_1109_TSTE_2025_3539894 crossref_primary_10_1038_s41598_025_98006_y crossref_primary_10_3390_jmse11061201 crossref_primary_10_1016_j_ijhydene_2024_08_013 crossref_primary_10_1016_j_apenergy_2024_122808 |
| Cites_doi | 10.1109/TEC.2005.847998 10.1016/j.jclepro.2021.128929 10.1109/TSTE.2019.2932103 10.1049/rpg2.12391 10.1109/MSP.2017.2743240 10.1109/TSG.2016.2597006 10.1016/S0378-7753(99)00430-9 10.1016/S0959-1524(02)00062-8 10.3390/su10072438 10.3390/pr8020154 10.1109/TEC.2017.2729881 10.1016/j.apenergy.2021.117541 10.1126/science.1204090 10.1109/TEC.2005.853756 10.1016/j.apenergy.2021.117542 10.3390/pr7120918 10.1109/TAC.2006.884959 10.1016/j.est.2021.103110 10.1109/MIE.2022.3148568 10.1109/TNNLS.2019.2905715 10.1109/TII.2010.2097601 10.1109/TSTE.2012.2210571 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023 |
| DBID | 97E RIA RIE AAYXX CITATION 7SP 7ST 7TB 8FD C1K FR3 H8D KR7 L7M SOI |
| DOI | 10.1109/TSTE.2022.3224179 |
| DatabaseName | IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998-Present IEEE Electronic Library (IEL) CrossRef Electronics & Communications Abstracts Environment Abstracts Mechanical & Transportation Engineering Abstracts Technology Research Database Environmental Sciences and Pollution Management Engineering Research Database Aerospace Database Civil Engineering Abstracts Advanced Technologies Database with Aerospace Environment Abstracts |
| DatabaseTitle | CrossRef Aerospace Database Civil Engineering Abstracts Technology Research Database Mechanical & Transportation Engineering Abstracts Electronics & Communications Abstracts Engineering Research Database Environment Abstracts Advanced Technologies Database with Aerospace Environmental Sciences and Pollution Management |
| DatabaseTitleList | Aerospace Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1949-3037 |
| EndPage | 16 |
| ExternalDocumentID | 10_1109_TSTE_2022_3224179 9961949 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: University of Western Australia grantid: BU/PG (00660 / 10300096) funderid: 10.13039/501100001801 |
| GroupedDBID | 0R~ 4.4 6IK 97E AAJGR AASAJ AAWTH ABQJQ ABVLG ACIWK AENEX AFRAH AGQYO AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS HZ~ IFIPE IPLJI JAVBF M43 O9- OCL P2P RIA RIE RNS 5VS AAYXX AGSQL CITATION EJD 7SP 7ST 7TB 8FD AARMG ABAZT C1K FR3 H8D KR7 L7M SOI |
| ID | FETCH-LOGICAL-c293t-341baf368e6602133c601692776f2d2a3a99895d6118cea5cb7a84d5b05fc9a13 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 15 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000911309200053&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1949-3029 |
| IngestDate | Mon Jun 30 08:42:42 EDT 2025 Tue Nov 18 21:32:40 EST 2025 Sat Nov 29 03:13:23 EST 2025 Tue Nov 25 14:44:28 EST 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 1 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c293t-341baf368e6602133c601692776f2d2a3a99895d6118cea5cb7a84d5b05fc9a13 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0003-2669-2095 0000-0002-4463-5927 0000-0002-0687-4038 0000-0001-9519-2321 0000-0001-9270-677X 0000-0002-9472-8785 0000-0001-7291-2242 0000-0002-3694-3276 0000-0003-0140-8887 |
| PQID | 2756558518 |
| PQPubID | 2040348 |
| PageCount | 16 |
| ParticipantIDs | crossref_citationtrail_10_1109_TSTE_2022_3224179 proquest_journals_2756558518 crossref_primary_10_1109_TSTE_2022_3224179 ieee_primary_9961949 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-01-01 |
| PublicationDateYYYYMMDD | 2023-01-01 |
| PublicationDate_xml | – month: 01 year: 2023 text: 2023-01-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | Piscataway |
| PublicationPlace_xml | – name: Piscataway |
| PublicationTitle | IEEE transactions on sustainable energy |
| PublicationTitleAbbrev | TSTE |
| PublicationYear | 2023 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref13 ref12 fujimoto (ref24) 0; 80 ref14 lillicrap (ref25) 2019 ref11 abbaker (ref10) 2019; 42 ref2 ref1 ref17 ref16 ref19 ref18 wang (ref27) 0 yu (ref15) 0 ref23 ref26 ref20 ref22 ref21 ref8 ref7 ref9 ref4 ref3 ref6 chatrattanawet (ref5) 2019; 7 |
| References_xml | – ident: ref17 doi: 10.1109/TEC.2005.847998 – start-page: 43 year: 0 ident: ref27 article-title: Discrete-time MPC with constraints publication-title: Proc Model Predictive Control Syst Des Implementation Using MATLAB – volume: 42 year: 2019 ident: ref10 article-title: Voltage control of solid oxide fuel cell power plant based on intelligent proportional integral-adaptive sliding mode control with anti-windup compensator publication-title: Trans Inst Meas Control – ident: ref13 doi: 10.1016/j.jclepro.2021.128929 – ident: ref6 doi: 10.1109/TSTE.2019.2932103 – ident: ref14 doi: 10.1049/rpg2.12391 – ident: ref23 doi: 10.1109/MSP.2017.2743240 – ident: ref9 doi: 10.1109/TSG.2016.2597006 – ident: ref16 doi: 10.1016/S0378-7753(99)00430-9 – volume: 80 start-page: 1587 year: 0 ident: ref24 article-title: Addressing function approximation error in actor-critic methods publication-title: Proc 35th Int Conf Mach Learn – ident: ref26 doi: 10.1016/S0959-1524(02)00062-8 – ident: ref4 doi: 10.3390/su10072438 – ident: ref11 doi: 10.3390/pr8020154 – ident: ref19 doi: 10.1109/TEC.2017.2729881 – ident: ref12 doi: 10.1016/j.apenergy.2021.117541 – ident: ref2 doi: 10.1126/science.1204090 – ident: ref18 doi: 10.1109/TEC.2005.853756 – ident: ref1 doi: 10.1016/j.apenergy.2021.117542 – volume: 7 year: 2019 ident: ref5 article-title: Design and implementation of the off-line robust model predictive control for solid oxide fuel cells publication-title: Process doi: 10.3390/pr7120918 – ident: ref20 doi: 10.1109/TAC.2006.884959 – ident: ref7 doi: 10.1016/j.est.2021.103110 – ident: ref8 doi: 10.1109/MIE.2022.3148568 – year: 2019 ident: ref25 article-title: Continuous control with deep reinforcement learning – start-page: 2570 year: 0 ident: ref15 article-title: Application of off-policy integral reinforcement learning for $H_{\infty }$, input constrained control of permanent magnet synchronous machine publication-title: Proc IEEE Appl Power Electron Conf Expo – ident: ref21 doi: 10.1109/TNNLS.2019.2905715 – ident: ref3 doi: 10.1109/TII.2010.2097601 – ident: ref22 doi: 10.1109/TSTE.2012.2210571 |
| SSID | ssj0000333630 |
| Score | 2.4416726 |
| Snippet | This paper proposes a new online integral reinforcement learning (IRL)-based control algorithm for the solid oxide fuel cell (SOFC) to overcome the... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 1 |
| SubjectTerms | Algorithms Computational modeling Computer applications Control methods Control theory DC Microgrid Distributed generation Fuel cells Fuel technology Hardware-In-the-Loop Integral Reinforcement Learning Learning Machine learning Mathematical models Microgrids Neural networks Optimal control Solid Oxide Fuel Cell Solid oxide fuel cells Training Tuning Twin Delayed Deep Deterministic Policy Gradient Voltage control |
| Title | A Novel Integral Reinforcement Learning-Based Control Method Assisted by Twin Delayed Deep Deterministic Policy Gradient for Solid Oxide Fuel Cell in DC Microgrid |
| URI | https://ieeexplore.ieee.org/document/9961949 https://www.proquest.com/docview/2756558518 |
| Volume | 14 |
| WOSCitedRecordID | wos000911309200053&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1949-3037 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000333630 issn: 1949-3029 databaseCode: RIE dateStart: 20100101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LT9tAEB4B6qEcaMtDpKVoDpwqHBxvbO8eaUJaDqQIgpSbtY8JimQlKCQU_g6_tLNrE0WiQuJira21vdK39jx29vsAjpS0iUuEioxKsogttOOWiCMnSeUt7YTSJohN5P2-HA7V5RocL_fCEFEoPqOmb4a1fDe1C58qO2HfnGNutQ7reZ5Ve7WW-ZRYCJEFaRHfJRJxoupFzFasTgbXgzMOBpOkKbzN8oVbK2Yo6Kq8-hkHC9P79L6xfYat2pPE0wr6L7BGk23YXOEX3IHnU-xPH6jE84oUosQrCkypNiQFsSZXvY1-si1z2KnK1vEiqEojI-fngEPzhIO_4wl2qdRPfN4luuNDVUcTiJ6xohfGX7NQQTZHfgde8zWHfx7HjrC34FF0qCzRP6eDF74O8HY2drtw0zsbdH5HtShDZNkzmEds9YweiUxSlrF_IIQNhC4JIzJi1LXQHMCp1GUcuVjSqTW5lm2XmjgdWaVbYg82JtMJ7QMKzd6YIqmsHLVNmumc3Ukykkxu20bqBsQvGBW2Ziz3whllESKXWBUe1sLDWtSwNuDH8pa7iq7jrc47HsdlxxrCBhy8TISi_qDvC8-Sn_olVPn1_3d9g49eib7KzhzAxny2oO_wwT7Mx_ezwzBX_wGEGebu |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1bT9RAFD5BNFEfQAXjIuh54MlY6HZ6mXmEhRUCuxqpCW_NXM6STZpdsuyi_B1-qWemZUOiMfGlmTbTdpJv2nOZM98HsKukTVwiVGRUkkdsoR23RBw5SaroaieUNkFsohgO5eWl-rYCn5d7YYgoFJ_Rnm-GtXw3tQufKttn35xjbvUEnmZpmsTNbq1lRiUWQuRBXMR3ikScqHYZsxur_fKiPOZwMEn2hLdavnTrkSEKyip__I6Djemv_9_oXsFa60viQQP-a1ihyRt4-YhhcAPuD3A4vaUaTxtaiBq_U-BKtSEtiC296lV0yNbMYa8pXMdB0JVGxs7PAofmDsuf4wkeUa3v-PyI6JoPTSVNoHrGhmAYv8xCDdkc-R14wdccfv01doT9BY-iR3WN_jk9HPhKwKvZ2G3Cj_5x2TuJWlmGyLJvMI_Y7hk9ErmkPGcPQQgbKF2SoshHjLsWmkM4lbmcYxdLOrOm0DJ1mYmzkVW6K97C6mQ6oXeAQrM_pkgqK0epyXJdsENJRpIpbGqk7kD8gFFlW85yL51RVyF2iVXlYa08rFULawc-LW-5bgg7_tV5w-O47NhC2IHth4lQtZ_0TeV58jO_iCq3_n7XR3h-Ug7Oq_PT4dl7eOF16ZtczTaszmcL2oFn9nY-vpl9CPP2N6rB6jU |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Novel+Integral+Reinforcement+Learning-Based+Control+Method+Assisted+by+Twin+Delayed+Deep+Deterministic+Policy+Gradient+for+Solid+Oxide+Fuel+Cell+in+DC+Microgrid&rft.jtitle=IEEE+transactions+on+sustainable+energy&rft.au=Liu%2C+Yulin&rft.au=Qie%2C+Tianhao&rft.au=Yu%2C+Yang&rft.au=Wang%2C+Yuxuan&rft.date=2023-01-01&rft.pub=IEEE&rft.issn=1949-3029&rft.spage=1&rft.epage=16&rft_id=info:doi/10.1109%2FTSTE.2022.3224179&rft.externalDocID=9961949 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1949-3029&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1949-3029&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1949-3029&client=summon |