Optimal operation of regional integrated energy system based on multi-agent deep deterministic policy gradient algorithm
The complex energy coupling and uncertainties of renewable energy and load make the dynamic scheduling of the integrated energy system (IES) very difficult. Therefore, an optimal operation method based on the multi-agent deep deterministic policy gradient algorithm (MADDPG) is proposed. Firstly, the...
Saved in:
| Published in: | Energy reports Vol. 8; pp. 932 - 939 |
|---|---|
| Main Authors: | , |
| Format: | Journal Article |
| Language: | English |
| Published: |
Elsevier Ltd
01.11.2022
Elsevier |
| Subjects: | |
| ISSN: | 2352-4847, 2352-4847 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | The complex energy coupling and uncertainties of renewable energy and load make the dynamic scheduling of the integrated energy system (IES) very difficult. Therefore, an optimal operation method based on the multi-agent deep deterministic policy gradient algorithm (MADDPG) is proposed. Firstly, the IES model is established, and the optimal scheduling problem is transformed into the Markov decision problem; Then, the action-space, state-space and reward function of agents are designed which control energy conversion equipment and energy storage equipment respectively. Furtherly, a multi-agent framework is established based on the MADDPG; Finally, a scheduling simulation example was carried out. The simulation results indicate that compared with the single agent, the multi-agent framework can improve the stability of training for agents and the ability to explore the optimal solution. |
|---|---|
| AbstractList | The complex energy coupling and uncertainties of renewable energy and load make the dynamic scheduling of the integrated energy system (IES) very difficult. Therefore, an optimal operation method based on the multi-agent deep deterministic policy gradient algorithm (MADDPG) is proposed. Firstly, the IES model is established, and the optimal scheduling problem is transformed into the Markov decision problem; Then, the action-space, state-space and reward function of agents are designed which control energy conversion equipment and energy storage equipment respectively. Furtherly, a multi-agent framework is established based on the MADDPG; Finally, a scheduling simulation example was carried out. The simulation results indicate that compared with the single agent, the multi-agent framework can improve the stability of training for agents and the ability to explore the optimal solution. |
| Author | Xu, Bohan Xiang, Yue |
| Author_xml | – sequence: 1 givenname: Bohan surname: Xu fullname: Xu, Bohan – sequence: 2 givenname: Yue surname: Xiang fullname: Xiang, Yue email: xiang@scu.edu.cn |
| BookMark | eNp9Uctq3TAQFSGFpkl-oCv9gN2xrOgB2ZTQRyCQTbMWsjR2dbEtI6kh_vvq5pZSushmZjjMOcyc84Gcr3FFQj520HbQiU-HFqc9tQwYa0G1IMQZuWD9DWu44vL8n_k9uc75AACdZsBFf0FeHrcSFjvTuGGyJcSVxpEmnOpU0bAWnCqOnuKKadpp3nPBhQ42V6xuL7_mEho74VqoR9xqKZiWsIZcgqNbnIPbadXw4bhi5ymmUH4uV-TdaOeM13_6JXn6-uXH3ffm4fHb_d3nh8bxDkrT2VGh5RJlr7xABSNXSnrpmB6G0fEb1BqFZ8ikGC0w6zQgQ6WlAK_l0F-S-5Ouj_ZgtlSfTbuJNphXIKbJ2FQvndGAGyzvuO3F4LmUTGtdi5LANFcCZNViJy2XYs4Jx796HZhjFOZgjlGYYxQGlKlRVJL6j-RCeXW6JBvmt6m3JypWg54DJpNdddGhDwldqR-Et-i_AWjHqTw |
| CitedBy_id | crossref_primary_10_3390_app13010274 crossref_primary_10_1016_j_ijhydene_2023_03_056 crossref_primary_10_3390_info16080662 crossref_primary_10_3390_pr13010268 crossref_primary_10_1016_j_cie_2024_110143 crossref_primary_10_1016_j_epsr_2022_108979 crossref_primary_10_1016_j_ijhydene_2024_09_164 crossref_primary_10_25729_esr_2025_02_0006 crossref_primary_10_1016_j_energy_2023_127087 crossref_primary_10_1016_j_psep_2024_09_123 |
| Cites_doi | 10.1016/j.apenergy.2015.12.089 10.1109/TSG.2020.3034827 10.1016/j.apenergy.2018.09.042 |
| ContentType | Journal Article |
| Copyright | 2022 The Author(s) |
| Copyright_xml | – notice: 2022 The Author(s) |
| DBID | 6I. AAFTH AAYXX CITATION DOA |
| DOI | 10.1016/j.egyr.2022.08.066 |
| DatabaseName | ScienceDirect Open Access Titles Elsevier:ScienceDirect:Open Access CrossRef DOAJ Directory of Open Access Journals |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: DOA name: DOAJ Directory of Open Access Journals url: https://www.doaj.org/ sourceTypes: Open Website |
| DeliveryMethod | fulltext_linktorsrc |
| EISSN | 2352-4847 |
| EndPage | 939 |
| ExternalDocumentID | oai_doaj_org_article_0cba414a36bd47729997298702948607 10_1016_j_egyr_2022_08_066 S2352484722014949 |
| GroupedDBID | 0R~ 0SF 4.4 457 5VS 6I. AACTN AAEDT AAEDW AAFTH AAIKJ AALRI AAXUO ABMAC ACGFS ADBBV ADEZE AEXQZ AFTJW AGHFR AITUG ALMA_UNASSIGNED_HOLDINGS AMRAJ BCNDV EBS EJD FDB GROUPED_DOAJ KQ8 M41 M~E NCXOZ O9- OK1 ROL SSZ AAYWO AAYXX ACVFH ADCNI ADVLN AEUPX AFJKZ AFPUW AIGII AKBMS AKRWK AKYEP APXCP CITATION |
| ID | FETCH-LOGICAL-c410t-1af8ea47e738d6e80f4887d7c29bbfc45e99e6d2e276fa02ac90e2e89760d97b3 |
| IEDL.DBID | DOA |
| ISICitedReferencesCount | 11 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000865597700051&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2352-4847 |
| IngestDate | Fri Oct 03 12:53:43 EDT 2025 Thu Nov 13 04:28:43 EST 2025 Tue Nov 18 21:53:30 EST 2025 Wed May 17 01:48:01 EDT 2023 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Integrated energy system Deep deterministic policy gradient algorithm Multi-agent scheduling |
| Language | English |
| License | This is an open access article under the CC BY-NC-ND license. |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c410t-1af8ea47e738d6e80f4887d7c29bbfc45e99e6d2e276fa02ac90e2e89760d97b3 |
| OpenAccessLink | https://doaj.org/article/0cba414a36bd47729997298702948607 |
| PageCount | 8 |
| ParticipantIDs | doaj_primary_oai_doaj_org_article_0cba414a36bd47729997298702948607 crossref_primary_10_1016_j_egyr_2022_08_066 crossref_citationtrail_10_1016_j_egyr_2022_08_066 elsevier_sciencedirect_doi_10_1016_j_egyr_2022_08_066 |
| PublicationCentury | 2000 |
| PublicationDate | November 2022 2022-11-00 2022-11-01 |
| PublicationDateYYYYMMDD | 2022-11-01 |
| PublicationDate_xml | – month: 11 year: 2022 text: November 2022 |
| PublicationDecade | 2020 |
| PublicationTitle | Energy reports |
| PublicationYear | 2022 |
| Publisher | Elsevier Ltd Elsevier |
| Publisher_xml | – name: Elsevier Ltd – name: Elsevier |
| References | Yang, Hu, Ai, Wu, Yang (b2) 2020; 130 An, Wang, Yuan (b8) 2020; 8 Munos, Stepleton, Harutyunyan, Bellemare (b9) 2016 Zhang, Li, Gao, Zhou (b7) 2019; 42 Zhang, Dehghanpour, Wang (b6) 2021; 12 Foruzan, Leen-Kiat, Soh, Sohrab, Asgarpoor (b4) 2018; 25 Wang, Lv, Peng, Song, Li, Xu (b1) 2018; 230 Ji Wang, Xu, Fang, Zhang (b5) 2019; 12 Mckenna, Thomson (b10) 2016; 165 Zhang, Zhang, Qiu (b3) 2020; 6 An (10.1016/j.egyr.2022.08.066_b8) 2020; 8 Zhang (10.1016/j.egyr.2022.08.066_b3) 2020; 6 Munos (10.1016/j.egyr.2022.08.066_b9) 2016 Mckenna (10.1016/j.egyr.2022.08.066_b10) 2016; 165 Zhang (10.1016/j.egyr.2022.08.066_b6) 2021; 12 Wang (10.1016/j.egyr.2022.08.066_b1) 2018; 230 Foruzan (10.1016/j.egyr.2022.08.066_b4) 2018; 25 Yang (10.1016/j.egyr.2022.08.066_b2) 2020; 130 Ji Wang (10.1016/j.egyr.2022.08.066_b5) 2019; 12 Zhang (10.1016/j.egyr.2022.08.066_b7) 2019; 42 |
| References_xml | – volume: 165 start-page: 445 year: 2016 end-page: 461 ident: b10 article-title: High-resolution stochastic integrated thermal–electrical domestic demand model publication-title: Appl Energy – volume: 12 start-page: 1048 year: 2021 end-page: 1062 ident: b6 article-title: Multi-agent safe policy learning for power management of networked microgrids publication-title: IEEE Trans Smart Grid – volume: 25 start-page: 876 year: 2018 end-page: 889 ident: b4 article-title: Reinforcement learning approach for optimal distributed energy management in a microgrid publication-title: IEEE Trans Power Syst – volume: 12 start-page: 2291 year: 2019 end-page: 2303 ident: b5 article-title: Real-time energy management of a microgrid using deep reinforcement learning publication-title: Energies – volume: 230 start-page: 1242 year: 2018 end-page: 1254 ident: b1 article-title: Modeling and optimal operation of community integrated energy systems: a case study from China publication-title: Appl Energy – volume: 42 start-page: 1077 year: 2019 end-page: 1089 ident: b7 article-title: Distributed optimal energy management for energy internet publication-title: IEEE Trans Ind Inf – volume: 6 start-page: 213 year: 2020 end-page: 225 ident: b3 article-title: Deep reinforcement learning for power system applications: An overview publication-title: CSEE J Power Energy Syst – volume: 8 start-page: 1166 year: 2020 end-page: 1179 ident: b8 article-title: Real-time optimal operation control of micro energy grid coupling with electricity- thermal -gas considering prosumer characteristics publication-title: IEEE Access – year: 2016 ident: b9 article-title: Safe and efficient off-policy reinforcement learning – volume: 130 start-page: 1102 year: 2020 end-page: 1114 ident: b2 article-title: Transactive energy supported economic operation for multi-energy complementary microgrids publication-title: IEEE Trans Smart Grid – volume: 165 start-page: 445 issue: 2016 year: 2016 ident: 10.1016/j.egyr.2022.08.066_b10 article-title: High-resolution stochastic integrated thermal–electrical domestic demand model publication-title: Appl Energy doi: 10.1016/j.apenergy.2015.12.089 – volume: 130 start-page: 1102 issue: 2020 year: 2020 ident: 10.1016/j.egyr.2022.08.066_b2 article-title: Transactive energy supported economic operation for multi-energy complementary microgrids publication-title: IEEE Trans Smart Grid – volume: 12 start-page: 1048 issue: 2021 year: 2021 ident: 10.1016/j.egyr.2022.08.066_b6 article-title: Multi-agent safe policy learning for power management of networked microgrids publication-title: IEEE Trans Smart Grid doi: 10.1109/TSG.2020.3034827 – volume: 12 start-page: 2291 issue: 2019 year: 2019 ident: 10.1016/j.egyr.2022.08.066_b5 article-title: Real-time energy management of a microgrid using deep reinforcement learning publication-title: Energies – volume: 25 start-page: 876 issue: 2018 year: 2018 ident: 10.1016/j.egyr.2022.08.066_b4 article-title: Reinforcement learning approach for optimal distributed energy management in a microgrid publication-title: IEEE Trans Power Syst – volume: 8 start-page: 1166 issue: 2020 year: 2020 ident: 10.1016/j.egyr.2022.08.066_b8 article-title: Real-time optimal operation control of micro energy grid coupling with electricity- thermal -gas considering prosumer characteristics publication-title: IEEE Access – volume: 230 start-page: 1242 issue: 2018 year: 2018 ident: 10.1016/j.egyr.2022.08.066_b1 article-title: Modeling and optimal operation of community integrated energy systems: a case study from China publication-title: Appl Energy doi: 10.1016/j.apenergy.2018.09.042 – volume: 42 start-page: 1077 issue: 2019 year: 2019 ident: 10.1016/j.egyr.2022.08.066_b7 article-title: Distributed optimal energy management for energy internet publication-title: IEEE Trans Ind Inf – volume: 6 start-page: 213 issue: 2020 year: 2020 ident: 10.1016/j.egyr.2022.08.066_b3 article-title: Deep reinforcement learning for power system applications: An overview publication-title: CSEE J Power Energy Syst – year: 2016 ident: 10.1016/j.egyr.2022.08.066_b9 |
| SSID | ssj0001920463 |
| Score | 2.3040328 |
| Snippet | The complex energy coupling and uncertainties of renewable energy and load make the dynamic scheduling of the integrated energy system (IES) very difficult.... |
| SourceID | doaj crossref elsevier |
| SourceType | Open Website Enrichment Source Index Database Publisher |
| StartPage | 932 |
| SubjectTerms | Deep deterministic policy gradient algorithm Integrated energy system Multi-agent scheduling |
| Title | Optimal operation of regional integrated energy system based on multi-agent deep deterministic policy gradient algorithm |
| URI | https://dx.doi.org/10.1016/j.egyr.2022.08.066 https://doaj.org/article/0cba414a36bd47729997298702948607 |
| Volume | 8 |
| WOSCitedRecordID | wos000865597700051&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAON databaseName: DOAJ Directory of Open Access Journals customDbUrl: eissn: 2352-4847 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001920463 issn: 2352-4847 databaseCode: DOA dateStart: 20150101 isFulltext: true titleUrlDefault: https://www.doaj.org/ providerName: Directory of Open Access Journals – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2352-4847 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001920463 issn: 2352-4847 databaseCode: M~E dateStart: 20150101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1LT9wwELYqxKEXBCpVl5d84IaiOo7Xdo6AQD0A7aGt9hb5MYFFsLta0gou_HZm7Oxqe1kuvViK5YytmUnmszTzDWPHEbyDKGwRVZCFiqoqrAy-wCcAqSrRJiLt31fm5saORvWPlVZflBOW6YGz4r6K4J0qlau0j4qgIFV64kVZyJr6J6U6cmHqlcvUfcYtRIWVOssN8Qz4D-4rZnJyF9y-EBmolJm_U_8TlRJ5_0pwWgk4l9tsq0eK_DSfcId9gMkn9vwdP_FHnJ7OINuOT1tO3RUIUfMl-UPkkIr6eGZq5hSsIsfVKYGwcFRQxSPADIecD5MIm_ks0QRzlEGZYB13D7fT-bi7e9xlvy4vfp5_K_rmCUVQpeiK0rUWnDJgKhs1WFK6NdEEWXvfBjWEugYdJUijWyekC7UACRbhiYi18dVntjGZTuAL4-Uw4C0ltqrVoHRU1pcRJSN0cMajeQesXCivCT2zODW4eGgWKWT3DSm8IYU31PVS6wE7Wb4zy7waa1efkU2WK4kTO02gpzS9pzTvecqADRcWbXp4kWEDihqv2Xzvf2y-zz6SyFzFeMA2uvkfOGSb4W83fpofJefF8fr14g3GOfP0 |
| linkProvider | Directory of Open Access Journals |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Optimal+operation+of+regional+integrated+energy+system+based+on+multi-agent+deep+deterministic+policy+gradient+algorithm&rft.jtitle=Energy+reports&rft.au=Xu%2C+Bohan&rft.au=Xiang%2C+Yue&rft.date=2022-11-01&rft.issn=2352-4847&rft.eissn=2352-4847&rft.volume=8&rft.spage=932&rft.epage=939&rft_id=info:doi/10.1016%2Fj.egyr.2022.08.066&rft.externalDBID=n%2Fa&rft.externalDocID=10_1016_j_egyr_2022_08_066 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2352-4847&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2352-4847&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2352-4847&client=summon |