Large-scale deep reinforcement learning method for energy management of power supply units considering regulation mileage payment
To improve automatic generation control (AGC) performance and reduce the wastage of regulation resources in interconnected grids including high-proportion renewable energy, a multi-area integrated AGC (MAI-AGC) framework is proposed to solve the coordination problem of secondary frequency regulation...
Uloženo v:
| Vydáno v: | Frontiers in energy research Ročník 11 |
|---|---|
| Hlavní autoři: | , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
Frontiers Media S.A
14.03.2024
|
| Témata: | |
| ISSN: | 2296-598X, 2296-598X |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | To improve automatic generation control (AGC) performance and reduce the wastage of regulation resources in interconnected grids including high-proportion renewable energy, a multi-area integrated AGC (MAI-AGC) framework is proposed to solve the coordination problem of secondary frequency regulation between different areas. In addition, a cocktail exploration multi-agent deep deterministic policy gradient (CE-MADDPG) algorithm is proposed as the framework algorithm. In this algorithm, the controller and power distributor of an area are combined into a single agent which can directly output the power generation command of different units. Moreover, the cocktail exploration strategy as well as various other techniques are introduced to improve the robustness of the framework. Through centralized training and decentralized execution, the proposed method can nonlinearly and adaptively derive the optimal coordinated control strategies for multiple agents and is verified on the two-area LFC model of southwest China and the four-area LFC model of the China Southern Grid (CSG). |
|---|---|
| AbstractList | To improve automatic generation control (AGC) performance and reduce the wastage of regulation resources in interconnected grids including high-proportion renewable energy, a multi-area integrated AGC (MAI-AGC) framework is proposed to solve the coordination problem of secondary frequency regulation between different areas. In addition, a cocktail exploration multi-agent deep deterministic policy gradient (CE-MADDPG) algorithm is proposed as the framework algorithm. In this algorithm, the controller and power distributor of an area are combined into a single agent which can directly output the power generation command of different units. Moreover, the cocktail exploration strategy as well as various other techniques are introduced to improve the robustness of the framework. Through centralized training and decentralized execution, the proposed method can nonlinearly and adaptively derive the optimal coordinated control strategies for multiple agents and is verified on the two-area LFC model of southwest China and the four-area LFC model of the China Southern Grid (CSG). |
| Author | Yang, Cheng Qian, Ting |
| Author_xml | – sequence: 1 givenname: Ting surname: Qian fullname: Qian, Ting – sequence: 2 givenname: Cheng surname: Yang fullname: Yang, Cheng |
| BookMark | eNp9kc9q3DAQh0VIIWmaF-hJL-Ct_tiyfAwhTQMLvSSQmxhLI1fBlozkpewxbx7vbiihh5xmmJnvO8zvKzmPKSIh3znbSKm7Hx5jHjaCCbnhcp2I9oxcCtGpqun08_mH_oJcl_LCGONSNDVnl-R1C3nAqlgYkTrEmWYM0adsccK40BEhxxAHOuHyJzm6bihGzMOeThBhOF0lT-f0FzMtu3ke93QXw1KoTbEEh_mAZxx2IywhRTqFVTognWF_gL-RLx7Ggtfv9Yo8_bx7vP1VbX_fP9zebCsrm3apsHNMqa7pm5Z7Jj13oLnS3HHm0ftOud4JzmurUVnfg-z7upYI0IrG1k7JK_Jw8roEL2bOYYK8NwmCOQ5SHgzkJdgRTd8hsxq04qquVdOAhU60yLX3oJzC1SVOLptTKRn9Px9n5pCJOWZiDpmY90xWSP8H2bAcX7JkCONn6BuXXJmF |
| CitedBy_id | crossref_primary_10_3390_biomimetics10060375 |
| Cites_doi | 10.3389/fenrg.2023.1259067 10.1016/j.automatica.2012.05.043 10.1109/TPWRS.2010.2102372 10.1061/(asce)ey.1943-7897.0000017 10.1016/j.advengsoft.2013.12.007 10.1109/TPWRS.2014.2357079 10.1016/j.apenergy.2020.116386 10.1109/TPWRS.2022.3166264 10.1016/j.ijhydene.2022.12.194 10.1016/j.enconman.2016.05.039 10.1109/TASE.2023.3263005 10.1016/j.ijepes.2021.107471 10.1109/JIOT.2023.3253693 10.1016/j.knosys.2015.12.022 10.1016/j.energy.2014.05.065 10.1016/j.enconman.2015.06.030 10.1061/(ASCE)EY.1943-7897.0000275 10.1016/j.engappai.2023.105818 10.1109/TPWRS.2022.3145907 10.1109/TSTE.2019.2958361 |
| ContentType | Journal Article |
| DBID | AAYXX CITATION DOA |
| DOI | 10.3389/fenrg.2023.1333827 |
| DatabaseName | CrossRef DOAJ Directory of Open Access Journals |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | CrossRef |
| Database_xml | – sequence: 1 dbid: DOA name: DOAJ Directory of Open Access Journals url: https://www.doaj.org/ sourceTypes: Open Website |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 2296-598X |
| ExternalDocumentID | oai_doaj_org_article_b9e0c8a861644655aca927e18ffa6d6e 10_3389_fenrg_2023_1333827 |
| GroupedDBID | 5VS 9T4 AAFWJ AAYXX ACGFS ADBBV AFPKN ALMA_UNASSIGNED_HOLDINGS BCNDV CITATION GROUPED_DOAJ KQ8 M~E OK1 |
| ID | FETCH-LOGICAL-c357t-e9d06695b571f03f1da81681d10feff96dbd2114c8e6cfba3bb443eaa725c4d63 |
| IEDL.DBID | DOA |
| ISICitedReferencesCount | 1 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001192109900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2296-598X |
| IngestDate | Fri Oct 03 12:51:49 EDT 2025 Sat Nov 29 03:07:10 EST 2025 Tue Nov 18 22:00:35 EST 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c357t-e9d06695b571f03f1da81681d10feff96dbd2114c8e6cfba3bb443eaa725c4d63 |
| OpenAccessLink | https://doaj.org/article/b9e0c8a861644655aca927e18ffa6d6e |
| ParticipantIDs | doaj_primary_oai_doaj_org_article_b9e0c8a861644655aca927e18ffa6d6e crossref_primary_10_3389_fenrg_2023_1333827 crossref_citationtrail_10_3389_fenrg_2023_1333827 |
| PublicationCentury | 2000 |
| PublicationDate | 2024-03-14 |
| PublicationDateYYYYMMDD | 2024-03-14 |
| PublicationDate_xml | – month: 03 year: 2024 text: 2024-03-14 day: 14 |
| PublicationDecade | 2020 |
| PublicationTitle | Frontiers in energy research |
| PublicationYear | 2024 |
| Publisher | Frontiers Media S.A |
| Publisher_xml | – name: Frontiers Media S.A |
| References | Li (B5) 2022; 136 Yu (B21) 2012; 48 Li (B6) 2021; 285 Qu (B12) 2022; 37 Yu (B19); 26 Yu (B18) 2016; 142 Li (B3); 120 Yu (B17) 2015; 30 Li (B4); 48 Mirjalili (B10) 2016; 96 Qu (B13) 2023; 38 Li (B8); 2023 Li (B7) 2023; 10 Xi (B14) 2020; 11 B9 Mirjalili (B11) 2014; 69 Bahrami (B1) 2014; 72 Xi (B15) 2015; 103 Huan (B2) 2023; 11 Yu (B20); 137 Xi (B16) 2016; 122 |
| References_xml | – ident: B9 – volume: 11 year: 2023 ident: B2 article-title: Multi-stage low-carbon planning of an integrated energy system considering demand response publication-title: Front. Energy Res. doi: 10.3389/fenrg.2023.1259067 – volume: 48 start-page: 2130 year: 2012 ident: B21 article-title: R(λ) imitation learning for automatic generation control of interconnected power grids publication-title: Automatica doi: 10.1016/j.automatica.2012.05.043 – volume: 26 start-page: 1272 ident: B19 article-title: Stochastic optimal relaxed automatic generation control in non-markov environment based on multi-step $Q(\lambda)$ learning publication-title: IEEE Trans. Power Syst. doi: 10.1109/TPWRS.2010.2102372 – volume: 137 start-page: 116 ident: B20 article-title: Stochastic optimal CPS relaxed control methodology for interconnected power systems using Q-learning method publication-title: J. doi: 10.1061/(asce)ey.1943-7897.0000017 – volume: 69 start-page: 46 year: 2014 ident: B11 article-title: Grey wolf optimizer publication-title: Adv. Eng. Softw. doi: 10.1016/j.advengsoft.2013.12.007 – volume: 30 start-page: 1669 year: 2015 ident: B17 article-title: Multi-agent correlated equilibrium Q(λ) learning for coordinated smart generation control of interconnected power grids publication-title: IEEE Trans. Power Syst. doi: 10.1109/TPWRS.2014.2357079 – volume: 285 start-page: 116386 year: 2021 ident: B6 article-title: Efficient experience replay based deep deterministic policy gradient for AGC dispatch in integrated energy system publication-title: Appl. Energ. doi: 10.1016/j.apenergy.2020.116386 – volume: 38 start-page: 818 year: 2023 ident: B13 article-title: Environmental-economic unit commitment with robust diffusion control of gas pollutants publication-title: IEEE Trans. Power Syst. doi: 10.1109/TPWRS.2022.3166264 – volume: 48 start-page: 14053 ident: B4 article-title: Optimal dual-model controller of solid oxide fuel cell output voltage using imitation distributed deep reinforcement learning publication-title: Int. J. Hydrogen Energy doi: 10.1016/j.ijhydene.2022.12.194 – volume: 122 start-page: 10 year: 2016 ident: B16 article-title: Wolf pack hunting strategy for automatic generation control of an islanding smart distribution network publication-title: Energ Convers. Manage doi: 10.1016/j.enconman.2016.05.039 – volume: 2023 start-page: 1 ident: B8 article-title: Brain-inspired deep meta-reinforcement learning for active coordinated fault-tolerant load frequency control of multi-area grids publication-title: IEEE Trans. Automation Sci. Eng. doi: 10.1109/TASE.2023.3263005 – volume: 136 start-page: 107471 year: 2022 ident: B5 article-title: Coordinated automatic generation control of interconnected power system with imitation guided exploration multi-agent deep reinforcement learning publication-title: Int. J. Elec Power doi: 10.1016/j.ijepes.2021.107471 – volume: 10 start-page: 12923 year: 2023 ident: B7 article-title: Evolutionary multi agent deep meta reinforcement learning method for swarm intelligence energy management of isolated multi area microgrid with internet of things publication-title: IEEE Internet Things J. doi: 10.1109/JIOT.2023.3253693 – volume: 96 start-page: 120 year: 2016 ident: B10 article-title: SCA: a Sine Cosine Algorithm for solving optimization problems publication-title: Knowl-Based Syst. doi: 10.1016/j.knosys.2015.12.022 – volume: 72 start-page: 434 year: 2014 ident: B1 article-title: Short term electric load forecasting by wavelet transform and grey model improved by PSO (particle swarm optimization) algorithm publication-title: Energy doi: 10.1016/j.energy.2014.05.065 – volume: 103 start-page: 82 year: 2015 ident: B15 article-title: A novel multi-agent decentralized win or learn fast policy hill-climbing with eligibility trace algorithm for smart generation control of interconnected complex power grids publication-title: Energ Convers. Manage doi: 10.1016/j.enconman.2015.06.030 – volume: 142 start-page: 04015012 year: 2016 ident: B18 article-title: Multiagent stochastic dynamic game for smart generation control publication-title: J. Energ Eng. doi: 10.1061/(ASCE)EY.1943-7897.0000275 – volume: 120 start-page: 105818 ident: B3 article-title: Distributed deep reinforcement learning-based gas supply system coordination management method for solid oxide fuel cell publication-title: Eng. Appl. Artif. Intell. doi: 10.1016/j.engappai.2023.105818 – volume: 37 start-page: 4497 year: 2022 ident: B12 article-title: Stochastic robust real-time power dispatch with wind uncertainty using difference-of-convexity optimization publication-title: IEEE Trans. Power Syst. doi: 10.1109/TPWRS.2022.3145907 – volume: 11 start-page: 2417 year: 2020 ident: B14 article-title: A novel multi-agent DDQN-AD method-based distributed strategy for automatic generation control of integrated energy systems publication-title: IEEE Trans. Sustain Energ doi: 10.1109/TSTE.2019.2958361 |
| SSID | ssj0001325410 |
| Score | 2.2574086 |
| Snippet | To improve automatic generation control (AGC) performance and reduce the wastage of regulation resources in interconnected grids including high-proportion... |
| SourceID | doaj crossref |
| SourceType | Open Website Enrichment Source Index Database |
| SubjectTerms | automatic generation control China Southern Grid frequency regulation mileage payment multi-agent deep deterministic policy gradient algorithm optimal coordinated control |
| Title | Large-scale deep reinforcement learning method for energy management of power supply units considering regulation mileage payment |
| URI | https://doaj.org/article/b9e0c8a861644655aca927e18ffa6d6e |
| Volume | 11 |
| WOSCitedRecordID | wos001192109900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAON databaseName: DOAJ Directory of Open Access Journals customDbUrl: eissn: 2296-598X dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001325410 issn: 2296-598X databaseCode: DOA dateStart: 20130101 isFulltext: true titleUrlDefault: https://www.doaj.org/ providerName: Directory of Open Access Journals – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2296-598X dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001325410 issn: 2296-598X databaseCode: M~E dateStart: 20130101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV07TxwxELYilIIUiCQgXkEu6KKF9dnrRwnoEEVAKRLpupUfYxQpHKd7INFE4p8z411OW5GGZotdz8r6PLZnrPH3MXaiA2gTTVM1MuRKRSGr4FWswEsXa0AXsrmITZjbWzuZuJ8DqS-qCevogTvgzoKDOlpvNcb1xPXlo3cjA8Lm7HXSQKsvRj2DZKqcrkhMfETd3ZLBLMydZRyOu1MSCz_FtExakpEZ7EQDwv6ys1xts60-JOTnXVc-sw8w_cI-DYgCv7LnH1SwXS0QUOAJYMbnUChPYznd4732wx3vBKE5fuFQbvXx-3V9C3_IfEaiaHxBUp5PfIXTecFjL9lJ5vNOmR7Hit_jcoGGfOafyHiH_b4a_7q8rnrxhCrKxiwrcAmjCdeExohcyyySJ4kNkUSdIWenU0iY_KloQcccvAxBKQnem1ETVdJyl21MH6awx7ipg21AapOtUKPobO1SUtgyi9qLZPaZeAWyjT2zOAlc_G0xwyDw2wJ-S-C3Pfj77PvaZtbxarzZ-oLGZ92SOLHLC_SUtveU9n-ecvAePzlkm9gxRVVoQh2xjeV8Bd_Yx_i4_LOYHxcnxOfNv_EL-JHnKA |
| linkProvider | Directory of Open Access Journals |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Large-scale+deep+reinforcement+learning+method+for+energy+management+of+power+supply+units+considering+regulation+mileage+payment&rft.jtitle=Frontiers+in+energy+research&rft.au=Qian%2C+Ting&rft.au=Yang%2C+Cheng&rft.date=2024-03-14&rft.issn=2296-598X&rft.eissn=2296-598X&rft.volume=11&rft_id=info:doi/10.3389%2Ffenrg.2023.1333827&rft.externalDBID=n%2Fa&rft.externalDocID=10_3389_fenrg_2023_1333827 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2296-598X&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2296-598X&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2296-598X&client=summon |