Multi-agent deep deterministic policy gradient algorithm for peer-to-peer energy trading considering distribution network constraints

In this paper, we investigate an energy cost minimization problem for prosumers participating in peer-to-peer energy trading. Due to (i) uncertainties caused by renewable energy generation and consumption, (ii) difficulties in developing an accurate and efficient energy trading model, and (iii) the...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	Applied energy Ročník 317; s. 119123
Hlavní autori:	Samende, Cephas, Cao, Jun, Fan, Zhong
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	Elsevier Ltd 01.07.2022
Predmet:	algorithms data collection Deep deterministic policy gradient energy energy costs issues and policy Markov decision process Multi-agent Peer-to-peer energy trading renewable energy sources Renewable generation Multi-agent Deep deterministic policy gradient Markov decision process Renewable generation Peer-to-peer energy trading
ISSN:	0306-2619, 1872-9118
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Abstract	In this paper, we investigate an energy cost minimization problem for prosumers participating in peer-to-peer energy trading. Due to (i) uncertainties caused by renewable energy generation and consumption, (ii) difficulties in developing an accurate and efficient energy trading model, and (iii) the need to satisfy distribution network constraints, it is challenging for prosumers to obtain optimal energy trading decisions that minimize their individual energy costs. To address the challenge, we first formulate the above problem as a Markov decision process and propose a multi-agent deep deterministic policy gradient algorithm to learn optimal energy trading decisions. To satisfy the distribution network constraints, we propose distribution network tariffs which we incorporate in the algorithm as incentives to incentivize energy trading decisions that help to satisfy the constraints and penalize the decisions that violate them. The proposed algorithm is model-free and allows the agents to learn the optimal energy trading decisions without having prior information about other agents in the network. Simulation results based on real-world datasets show the effectiveness and robustness of the proposed algorithm. •Deep reinforcement learning-based algorithm for P2P energy trading considering network constraints is proposed.•The resulting trading strategy minimizes the total energy cost of prosumers.•Distribution Network Tariffs (DNT) are proposed to manage the network constraints.•Results show the effectiveness of the proposed algorithm and that DNTs improves voltage regulation, reduces network losses and peak congestion.
AbstractList	In this paper, we investigate an energy cost minimization problem for prosumers participating in peer-to-peer energy trading. Due to (i) uncertainties caused by renewable energy generation and consumption, (ii) difficulties in developing an accurate and efficient energy trading model, and (iii) the need to satisfy distribution network constraints, it is challenging for prosumers to obtain optimal energy trading decisions that minimize their individual energy costs. To address the challenge, we first formulate the above problem as a Markov decision process and propose a multi-agent deep deterministic policy gradient algorithm to learn optimal energy trading decisions. To satisfy the distribution network constraints, we propose distribution network tariffs which we incorporate in the algorithm as incentives to incentivize energy trading decisions that help to satisfy the constraints and penalize the decisions that violate them. The proposed algorithm is model-free and allows the agents to learn the optimal energy trading decisions without having prior information about other agents in the network. Simulation results based on real-world datasets show the effectiveness and robustness of the proposed algorithm. •Deep reinforcement learning-based algorithm for P2P energy trading considering network constraints is proposed.•The resulting trading strategy minimizes the total energy cost of prosumers.•Distribution Network Tariffs (DNT) are proposed to manage the network constraints.•Results show the effectiveness of the proposed algorithm and that DNTs improves voltage regulation, reduces network losses and peak congestion. In this paper, we investigate an energy cost minimization problem for prosumers participating in peer-to-peer energy trading. Due to (i) uncertainties caused by renewable energy generation and consumption, (ii) difficulties in developing an accurate and efficient energy trading model, and (iii) the need to satisfy distribution network constraints, it is challenging for prosumers to obtain optimal energy trading decisions that minimize their individual energy costs. To address the challenge, we first formulate the above problem as a Markov decision process and propose a multi-agent deep deterministic policy gradient algorithm to learn optimal energy trading decisions. To satisfy the distribution network constraints, we propose distribution network tariffs which we incorporate in the algorithm as incentives to incentivize energy trading decisions that help to satisfy the constraints and penalize the decisions that violate them. The proposed algorithm is model-free and allows the agents to learn the optimal energy trading decisions without having prior information about other agents in the network. Simulation results based on real-world datasets show the effectiveness and robustness of the proposed algorithm.
ArticleNumber	119123
Author	Samende, Cephas Fan, Zhong Cao, Jun
Author_xml	– sequence: 1 givenname: Cephas surname: Samende fullname: Samende, Cephas email: c.samende@keele.ac.uk organization: School of Computing and Mathematics, Keele University, UK – sequence: 2 givenname: Jun surname: Cao fullname: Cao, Jun organization: Environmental Research and Innovation Department, LIST, Luxembourg – sequence: 3 givenname: Zhong surname: Fan fullname: Fan, Zhong organization: School of Computing and Mathematics, Keele University, UK
BookMark	eNqFkLFuFDEQhi0UJC4Jr4Bc0uzhsS97Z4kCFBESKYgGastrzy5z7NmL7QPdA_DeWWdJQ5PG4-L7f8185-wsxICMvQGxBgHtu_3aThgwDae1FFKuATRI9YKtYLeVjQbYnbGVUKJtZAv6FTvPeS-EkCDFiv39chwLNXbAULhHnOanYDpQoFzI8SmO5E58SNZTRew4xETlx4H3MfEJMTUlNnXyZQdeKhoG7mLI5DHVv5_LEnXHQjHwgOVPTD8fgRmmUPIle9nbMePrf_OCfb_59O36trn_-vnu-uN945RWpVEKYOO61vuu3alO-Cu86qCzGqHvvEZrpXYopEeNWloA7HyvncSttRutNuqCvV16pxR_HTEXc6DscBxtwHjMRrbtrFRudxV9v6AuxZwT9sZRsfWAuvNoQJhq3-zNk31T7ZvF_hxv_4tPiQ42nZ4PfliCOHv4TZhMdrN5h54SumJ8pOcqHgDXvKsf
CitedBy_id	crossref_primary_10_1016_j_egyr_2022_09_046 crossref_primary_10_1109_TSG_2023_3277330 crossref_primary_10_1016_j_engappai_2023_106388 crossref_primary_10_1016_j_rcim_2025_103015 crossref_primary_10_1016_j_autcon_2024_105702 crossref_primary_10_1007_s00202_024_02732_x crossref_primary_10_1016_j_rser_2023_113435 crossref_primary_10_1016_j_jmsy_2024_08_021 crossref_primary_10_3390_app13137838 crossref_primary_10_1016_j_apenergy_2024_124641 crossref_primary_10_3390_electronics13071281 crossref_primary_10_1155_2023_4295384 crossref_primary_10_1016_j_apenergy_2024_123518 crossref_primary_10_1016_j_eswa_2023_120663 crossref_primary_10_1016_j_segan_2024_101570 crossref_primary_10_1109_ACCESS_2022_3224460 crossref_primary_10_1109_ACCESS_2023_3287323 crossref_primary_10_1016_j_jclepro_2023_136511 crossref_primary_10_3390_en17061423 crossref_primary_10_1016_j_asej_2023_102466 crossref_primary_10_1109_ACCESS_2025_3609317 crossref_primary_10_3390_buildings13123096 crossref_primary_10_1109_JIOT_2025_3541350 crossref_primary_10_1186_s42162_022_00235_2 crossref_primary_10_1016_j_apenergy_2023_120827 crossref_primary_10_1109_JIOT_2024_3377600 crossref_primary_10_1109_JPROC_2023_3303358 crossref_primary_10_1109_TII_2023_3348823 crossref_primary_10_1016_j_apenergy_2025_125485 crossref_primary_10_1016_j_engappai_2024_107847 crossref_primary_10_3390_en16196770 crossref_primary_10_1016_j_apenergy_2023_122396 crossref_primary_10_1016_j_apenergy_2023_121144 crossref_primary_10_1109_JIOT_2025_3535767 crossref_primary_10_1016_j_adhoc_2025_103996 crossref_primary_10_3390_jmse11061201 crossref_primary_10_1016_j_segan_2024_101344 crossref_primary_10_1016_j_ijepes_2023_109753 crossref_primary_10_1109_TSMC_2023_3250620 crossref_primary_10_1016_j_energy_2025_136178 crossref_primary_10_1177_14727978251361392 crossref_primary_10_1109_TSG_2022_3222323 crossref_primary_10_1016_j_energy_2025_136075 crossref_primary_10_1080_15567036_2025_2548357
Cites_doi	10.1016/j.apenergy.2021.116940 10.1109/JIOT.2019.2941498 10.1109/TSG.2015.2495145 10.1109/TPWRS.2017.2715983 10.1186/s41601-021-00209-8 10.1016/j.apenergy.2017.08.136 10.3390/en13205359 10.1016/j.apenergy.2013.08.062 10.1016/j.apenergy.2018.05.097 10.24963/ijcai.2021/401 10.1109/TSG.2018.2878445 10.1109/TSG.2019.2963238 10.1103/PhysRev.36.823 10.1109/TSG.2020.2969657 10.1038/s41560-017-0075-y 10.1038/nature14236 10.1109/TSG.2020.2986333 10.1109/TPWRS.2017.2649558 10.1109/TPWRS.2017.2767632 10.1016/j.eng.2020.06.002 10.3390/en11061560 10.1109/TEC.2006.874230 10.1109/TPWRS.2018.2834472 10.1016/j.esr.2019.100418 10.1109/TPWRS.2019.2961330 10.1109/TPWRS.2013.2255317 10.1109/TSG.2016.2614988 10.1109/TSG.2021.3103917 10.1109/ACCESS.2018.2876652 10.1109/TII.2020.2980160 10.1109/TSG.2017.2673860
ContentType	Journal Article
Copyright	2022 The Author(s)
Copyright_xml	– notice: 2022 The Author(s)
DBID	6I. AAFTH AAYXX CITATION 7S9 L.6
DOI	10.1016/j.apenergy.2022.119123
DatabaseName	ScienceDirect Open Access Titles Elsevier:ScienceDirect:Open Access CrossRef AGRICOLA AGRICOLA - Academic
DatabaseTitle	CrossRef AGRICOLA AGRICOLA - Academic
DatabaseTitleList	AGRICOLA
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering Environmental Sciences
EISSN	1872-9118
ExternalDocumentID	10_1016_j_apenergy_2022_119123 S0306261922005025
GroupedDBID	--K --M .~1 0R~ 1B1 1~. 1~5 23M 4.4 457 4G. 5GY 5VS 6I. 7-5 71M 8P~ 9JN AABNK AACTN AAEDT AAEDW AAFTH AAHCO AAIAV AAIKJ AAKOC AALRI AAOAW AAQFI AARJD AAXUO ABJNI ABMAC ABYKQ ACDAQ ACGFS ACRLP ADBBV ADEZE ADTZH AEBSH AECPX AEKER AENEX AFKWA AFTJW AGHFR AGUBO AGYEJ AHHHB AHIDL AHJVU AIEXJ AIKHN AITUG AJOXV ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AXJTR BELTK BJAXD BKOJK BLXMC CS3 EBS EFJIC EFLBG EO8 EO9 EP2 EP3 FDB FIRID FNPLU FYGXN G-Q GBLVA IHE J1W JARJE JJJVA KOM LY6 M41 MO0 N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. Q38 ROL RPZ SDF SDG SES SPC SPCBC SSR SST SSZ T5K TN5 ~02 ~G- 9DU AAHBH AAQXK AATTM AAXKI AAYWO AAYXX ABEFU ABFNM ABWVN ABXDB ACLOT ACNNM ACRPL ACVFH ADCNI ADMUD ADNMO AEIPS AEUPX AFJKZ AFPUW AGQPQ AIGII AIIUN AKBMS AKRWK AKYEP ANKPU APXCP ASPBG AVWKF AZFZN CITATION EFKBS EJD FEDTE FGOYB G-2 HVGLF HZ~ R2- SAC SEW WUQ ZY4 ~HD 7S9 L.6
ID	FETCH-LOGICAL-c393t-33114cb6ddb683b0d5e5b1ba9e1fbd9eaa29ce02de9e92a11ebdf9c2e7aa49343
ISICitedReferencesCount	51
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000799979200006&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0306-2619
IngestDate	Sun Nov 09 09:32:02 EST 2025 Sat Nov 29 07:22:23 EST 2025 Tue Nov 18 21:19:31 EST 2025 Fri Feb 23 02:39:42 EST 2024
IsDoiOpenAccess	true
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Keywords	Multi-agent Deep deterministic policy gradient Markov decision process Renewable generation Peer-to-peer energy trading
Language	English
License	This is an open access article under the CC BY-NC-ND license.
LinkModel	OpenURL
MergedId	FETCHMERGED-LOGICAL-c393t-33114cb6ddb683b0d5e5b1ba9e1fbd9eaa29ce02de9e92a11ebdf9c2e7aa49343
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23
OpenAccessLink	https://dx.doi.org/10.1016/j.apenergy.2022.119123
PQID	2661012784
PQPubID	24069
ParticipantIDs	proquest_miscellaneous_2661012784 crossref_citationtrail_10_1016_j_apenergy_2022_119123 crossref_primary_10_1016_j_apenergy_2022_119123 elsevier_sciencedirect_doi_10_1016_j_apenergy_2022_119123
PublicationCentury	2000
PublicationDate	2022-07-01 2022-07-00 20220701
PublicationDateYYYYMMDD	2022-07-01
PublicationDate_xml	– month: 07 year: 2022 text: 2022-07-01 day: 01
PublicationDecade	2020
PublicationTitle	Applied energy
PublicationYear	2022
Publisher	Elsevier Ltd
Publisher_xml	– name: Elsevier Ltd
References	Chen, Bu, Liu, Kang, Yu, Han (b29) 2021 Farivar, Low (b38) 2013; 28 Kim, Dvorkin (b19) 2019; 35 Qiu, Ye, Papadaskalopoulos, Strbac (b28) 2021; 292 Dzobo, Malila, Sithole (b6) 2021; 6 Chen, Su (b13) 2018; 6 Morstyn, McCulloch (b20) 2018; 34 Tushar, Saha, Yuen, Smith, Poor (b5) 2020; 11 Ye, Tang, Wang, Zhang, Strbac (b31) 2021; 12 Qiu, Zhao, Yang, Dong (b3) 2018; 33 Qiu D, Wang J, Wang J, Strbac G. Multi-agent reinforcement learning for automated peer-to-peer energy trading in double-side auction market. In: Proceedings of the thirtieth international joint conference on artificial intelligence. 2021, p. 2913–20. Long, Wu, Zhou, Jenkins (b8) 2018; 226 Bai, Wang, Wang, Chen, Li (b39) 2017; 33 Uhlenbeck, Ornstein (b44) 1930; 36 Morstyn, Teytelboym, Hepburn, McCulloch (b18) 2019; 11 Brockman, Cheung, Pettersson, Schneider, Schulman, Tang (b46) 2016 Morstyn, Farrell, Darby, McCulloch (b4) 2018; 3 Abdella, Shuaib (b10) 2018; 11 Lowe, Wu, Tamar, Harb, Abbeel, Mordatch (b32) 2017 Han, Han, Aki (b36) 2014; 113 Mnih, Kavukcuoglu, Silver, Rusu, Veness, Bellemare (b22) 2015; 518 Paszke, Gross, Massa, Lerer, Bradbury, Chanan (b45) 2019 Kim, Lee (b25) 2020; 13 Silver, Lever, Heess, Degris, Wierstra, Riedmiller (b42) 2014 Lu, Xiao, Xiao, Dai, Peng, Poor (b12) 2019; 6 Abrishambaf, Lezama, Faria, Vale (b7) 2019; 26 Olivella-Rosell, Bullich-Massagué, Aragüés-Peñalba, Sumper, Ottesen, Vidal-Clos (b34) 2018; 210 Chen, Bu (b24) 2019 Farivar, Neal, Clarke, Low (b40) 2012 Wood, Wollenberg, Sheblé (b2) 2013 Gao, Wen, Wu, Wang (b15) 2020 Sutton, Barto (b23) 2018 Cao, Harrold, Fan, Morstyn, Healey, Li (b26) 2020; 11 Guerrero, Chapman, Verbič (b17) 2018; 10 Papavasiliou (b33) 2017; 9 Wang, Huang (b9) 2016; 9 Wang, Liu, Jia, Zhang, Chen, Huang (b11) 2020 Bi, Shu, Dong, Yang (b14) 2020 Paudel, Khorasany, Gooi (b21) 2020; 17 Kim, Zhang, Van Der Schaar, Lee (b16) 2015; 7 Yu, Qin, Zhang, Shen, Jiang, Guan (b27) 2020 Esram, Chapman (b35) 2007; 22 Liu, Yu, Wang, Li, Ma, Lei (b37) 2017; 32 Lillicrap, Hunt, Pritzel, Heess, Erez, Tassa (b43) 2015 Littman (b41) 1994 Zhou, Wu, Long, Ming (b1) 2020; 6 Wang (10.1016/j.apenergy.2022.119123_b11) 2020 Farivar (10.1016/j.apenergy.2022.119123_b40) 2012 Kim (10.1016/j.apenergy.2022.119123_b16) 2015; 7 Bi (10.1016/j.apenergy.2022.119123_b14) 2020 Silver (10.1016/j.apenergy.2022.119123_b42) 2014 Qiu (10.1016/j.apenergy.2022.119123_b3) 2018; 33 Abdella (10.1016/j.apenergy.2022.119123_b10) 2018; 11 Bai (10.1016/j.apenergy.2022.119123_b39) 2017; 33 Tushar (10.1016/j.apenergy.2022.119123_b5) 2020; 11 Farivar (10.1016/j.apenergy.2022.119123_b38) 2013; 28 Littman (10.1016/j.apenergy.2022.119123_b41) 1994 Cao (10.1016/j.apenergy.2022.119123_b26) 2020; 11 Esram (10.1016/j.apenergy.2022.119123_b35) 2007; 22 Kim (10.1016/j.apenergy.2022.119123_b19) 2019; 35 Mnih (10.1016/j.apenergy.2022.119123_b22) 2015; 518 Morstyn (10.1016/j.apenergy.2022.119123_b4) 2018; 3 Long (10.1016/j.apenergy.2022.119123_b8) 2018; 226 Abrishambaf (10.1016/j.apenergy.2022.119123_b7) 2019; 26 Chen (10.1016/j.apenergy.2022.119123_b24) 2019 Chen (10.1016/j.apenergy.2022.119123_b13) 2018; 6 Morstyn (10.1016/j.apenergy.2022.119123_b18) 2019; 11 Olivella-Rosell (10.1016/j.apenergy.2022.119123_b34) 2018; 210 Uhlenbeck (10.1016/j.apenergy.2022.119123_b44) 1930; 36 Lu (10.1016/j.apenergy.2022.119123_b12) 2019; 6 Lillicrap (10.1016/j.apenergy.2022.119123_b43) 2015 Paszke (10.1016/j.apenergy.2022.119123_b45) 2019 Guerrero (10.1016/j.apenergy.2022.119123_b17) 2018; 10 Ye (10.1016/j.apenergy.2022.119123_b31) 2021; 12 Chen (10.1016/j.apenergy.2022.119123_b29) 2021 Dzobo (10.1016/j.apenergy.2022.119123_b6) 2021; 6 Gao (10.1016/j.apenergy.2022.119123_b15) 2020 Paudel (10.1016/j.apenergy.2022.119123_b21) 2020; 17 Kim (10.1016/j.apenergy.2022.119123_b25) 2020; 13 Sutton (10.1016/j.apenergy.2022.119123_b23) 2018 Papavasiliou (10.1016/j.apenergy.2022.119123_b33) 2017; 9 Zhou (10.1016/j.apenergy.2022.119123_b1) 2020; 6 Yu (10.1016/j.apenergy.2022.119123_b27) 2020 10.1016/j.apenergy.2022.119123_b30 Morstyn (10.1016/j.apenergy.2022.119123_b20) 2018; 34 Wang (10.1016/j.apenergy.2022.119123_b9) 2016; 9 Brockman (10.1016/j.apenergy.2022.119123_b46) 2016 Lowe (10.1016/j.apenergy.2022.119123_b32) 2017 Han (10.1016/j.apenergy.2022.119123_b36) 2014; 113 Wood (10.1016/j.apenergy.2022.119123_b2) 2013 Qiu (10.1016/j.apenergy.2022.119123_b28) 2021; 292 Liu (10.1016/j.apenergy.2022.119123_b37) 2017; 32
References_xml	– volume: 10 start-page: 5163 year: 2018 end-page: 5173 ident: b17 article-title: Decentralized P2P energy trading under network constraints in a low-voltage network publication-title: IEEE Trans Smart Grid – volume: 11 start-page: 1560 year: 2018 ident: b10 article-title: Peer to peer distributed energy trading in smart grids: A survey publication-title: Energies – volume: 226 start-page: 261 year: 2018 end-page: 276 ident: b8 article-title: Peer-to-peer energy sharing through a two-stage aggregated battery control in a community microgrid publication-title: Appl Energy – start-page: 1 year: 2019 end-page: 5 ident: b24 article-title: Realistic peer-to-peer energy trading model for microgrids using deep reinforcement learning publication-title: 2019 IEEE PES innovative smart grid technologies Europe – volume: 35 start-page: 2716 year: 2019 end-page: 2725 ident: b19 article-title: A P2P-dominant distribution system architecture publication-title: IEEE Trans Power Syst – volume: 28 start-page: 2554 year: 2013 end-page: 2564 ident: b38 article-title: Branch flow model: Relaxations and convexification—Part I publication-title: IEEE Trans Power Syst – volume: 113 start-page: 1100 year: 2014 end-page: 1108 ident: b36 article-title: A practical battery wear model for electric vehicle charging applications publication-title: Appl Energy – volume: 32 start-page: 3569 year: 2017 end-page: 3583 ident: b37 article-title: Energy-sharing model with price-based demand response for microgrids of peer-to-peer prosumers publication-title: IEEE Trans Power Syst – volume: 11 start-page: 3185 year: 2020 end-page: 3200 ident: b5 article-title: Peer-to-peer trading in electricity networks: An overview publication-title: IEEE Trans Smart Grid – volume: 26 year: 2019 ident: b7 article-title: Towards transactive energy systems: An analysis on current trends publication-title: Energy Strategy Rev – start-page: 38 year: 2020 end-page: 41 ident: b14 article-title: Real-time energy management of microgrid using reinforcement learning publication-title: 2020 19th International symposium on distributed computing and applications for business engineering and science – year: 2020 ident: b27 article-title: Deep reinforcement learning for smart building energy management: A survey – volume: 33 start-page: 4061 year: 2017 end-page: 4073 ident: b39 article-title: Distribution locational marginal pricing (DLMP) for congestion management and voltage support publication-title: IEEE Trans Power Syst – year: 2021 ident: b29 article-title: Peer-to-peer energy trading and energy conversion in interconnected multi-energy microgrids using multi-agent deep reinforcement learning publication-title: IEEE Trans Smart Grid – year: 2019 ident: b45 article-title: Pytorch: An imperative style, high-performance deep learning library – volume: 7 start-page: 2187 year: 2015 end-page: 2198 ident: b16 article-title: Dynamic pricing and energy consumption scheduling with reinforcement learning publication-title: IEEE Trans Smart Grid – volume: 36 start-page: 823 year: 1930 ident: b44 article-title: On the theory of the Brownian motion publication-title: Phys Rev – year: 2018 ident: b23 article-title: Reinforcement learning: An introduction – reference: Qiu D, Wang J, Wang J, Strbac G. Multi-agent reinforcement learning for automated peer-to-peer energy trading in double-side auction market. In: Proceedings of the thirtieth international joint conference on artificial intelligence. 2021, p. 2913–20. – start-page: 157 year: 1994 end-page: 163 ident: b41 article-title: Markov games as a framework for multi-agent reinforcement learning publication-title: Machine learning proceedings 1994 – year: 2017 ident: b32 article-title: Multi-agent actor-critic for mixed cooperative-competitive environments – volume: 33 start-page: 1970 year: 2018 end-page: 1980 ident: b3 article-title: Optimal scheduling for prosumers in coupled transactive power and gas systems publication-title: IEEE Trans Power Syst – start-page: 1 year: 2012 end-page: 7 ident: b40 article-title: Optimal inverter VAR control in distribution systems with high PV penetration publication-title: 2012 IEEE power and energy society general meeting – volume: 9 start-page: 2647 year: 2016 end-page: 2657 ident: b9 article-title: Incentivizing energy trading for interconnected microgrids publication-title: IEEE Trans Smart Grid – volume: 6 start-page: 1 year: 2021 end-page: 11 ident: b6 article-title: Proposed framework for blockchain technology in a decentralised energy network publication-title: Protection and Control of Modern Power Systems – volume: 518 start-page: 529 year: 2015 end-page: 533 ident: b22 article-title: Human-level control through deep reinforcement learning publication-title: Nature – start-page: 387 year: 2014 end-page: 395 ident: b42 article-title: Deterministic policy gradient algorithms publication-title: International conference on machine learning – year: 2013 ident: b2 article-title: Power generation, operation, and control – volume: 9 start-page: 4872 year: 2017 end-page: 4882 ident: b33 article-title: Analysis of distribution locational marginal prices publication-title: IEEE Trans Smart Grid – volume: 11 start-page: 4513 year: 2020 end-page: 4521 ident: b26 article-title: Deep reinforcement learning-based energy storage arbitrage with accurate lithium-ion battery degradation model publication-title: IEEE Trans Smart Grid – volume: 292 year: 2021 ident: b28 article-title: Scalable coordinated management of peer-to-peer energy trading: A multi-cluster deep reinforcement learning approach publication-title: Appl Energy – volume: 210 start-page: 881 year: 2018 end-page: 895 ident: b34 article-title: Optimization problem for meeting distribution system operator requests in local flexibility markets with distributed energy resources publication-title: Appl Energy – volume: 6 start-page: 62806 year: 2018 end-page: 62814 ident: b13 article-title: Local energy trading behavior modeling with deep reinforcement learning publication-title: IEEE Access – volume: 34 start-page: 4005 year: 2018 end-page: 4014 ident: b20 article-title: Multiclass energy management for peer-to-peer energy trading driven by prosumer preferences publication-title: IEEE Trans Power Syst – year: 2015 ident: b43 article-title: Continuous control with deep reinforcement learning – volume: 13 start-page: 5359 year: 2020 ident: b25 article-title: Automatic P2P energy trading model based on reinforcement learning using long short-term delayed reward publication-title: Energies – volume: 12 start-page: 5185 year: 2021 end-page: 5200 ident: b31 article-title: A scalable privacy-preserving multi-agent deep reinforcement learning approach for large-scale peer-to-peer transactive energy trading publication-title: IEEE Trans Smart Grid – year: 2016 ident: b46 article-title: Openai gym – volume: 22 start-page: 439 year: 2007 end-page: 449 ident: b35 article-title: Comparison of photovoltaic array maximum power point tracking techniques publication-title: IEEE Trans Energy Convers – volume: 6 start-page: 739 year: 2020 end-page: 753 ident: b1 article-title: State-of-the-art analysis and perspectives for peer-to-peer energy trading publication-title: Engineering – year: 2020 ident: b11 article-title: Peer-to-peer electricity transaction decision of user-side smart energy system based on SARSA reinforcement learning method publication-title: CSEE J Power Energy Syst – volume: 6 start-page: 10728 year: 2019 end-page: 10737 ident: b12 article-title: Reinforcement learning-based microgrid energy trading with a reduced power plant schedule publication-title: IEEE Internet Things J – volume: 11 start-page: 3095 year: 2019 end-page: 3106 ident: b18 article-title: Integrating P2P energy trading with probabilistic distribution locational marginal pricing publication-title: IEEE Trans Smart Grid – year: 2020 ident: b15 article-title: Distributed energy trading and scheduling among microgrids via multiagent reinforcement learning – volume: 17 start-page: 1111 year: 2020 end-page: 1121 ident: b21 article-title: Decentralized local energy trading in microgrids with voltage management publication-title: IEEE Trans Ind Inf – volume: 3 start-page: 94 year: 2018 end-page: 101 ident: b4 article-title: Using peer-to-peer energy-trading platforms to incentivize prosumers to form federated power plants publication-title: Nat Energy – volume: 292 year: 2021 ident: 10.1016/j.apenergy.2022.119123_b28 article-title: Scalable coordinated management of peer-to-peer energy trading: A multi-cluster deep reinforcement learning approach publication-title: Appl Energy doi: 10.1016/j.apenergy.2021.116940 – volume: 6 start-page: 10728 issue: 6 year: 2019 ident: 10.1016/j.apenergy.2022.119123_b12 article-title: Reinforcement learning-based microgrid energy trading with a reduced power plant schedule publication-title: IEEE Internet Things J doi: 10.1109/JIOT.2019.2941498 – volume: 7 start-page: 2187 issue: 5 year: 2015 ident: 10.1016/j.apenergy.2022.119123_b16 article-title: Dynamic pricing and energy consumption scheduling with reinforcement learning publication-title: IEEE Trans Smart Grid doi: 10.1109/TSG.2015.2495145 – volume: 33 start-page: 1970 issue: 2 year: 2018 ident: 10.1016/j.apenergy.2022.119123_b3 article-title: Optimal scheduling for prosumers in coupled transactive power and gas systems publication-title: IEEE Trans Power Syst doi: 10.1109/TPWRS.2017.2715983 – volume: 6 start-page: 1 issue: 1 year: 2021 ident: 10.1016/j.apenergy.2022.119123_b6 article-title: Proposed framework for blockchain technology in a decentralised energy network publication-title: Protection and Control of Modern Power Systems doi: 10.1186/s41601-021-00209-8 – year: 2018 ident: 10.1016/j.apenergy.2022.119123_b23 – volume: 210 start-page: 881 year: 2018 ident: 10.1016/j.apenergy.2022.119123_b34 article-title: Optimization problem for meeting distribution system operator requests in local flexibility markets with distributed energy resources publication-title: Appl Energy doi: 10.1016/j.apenergy.2017.08.136 – volume: 13 start-page: 5359 issue: 20 year: 2020 ident: 10.1016/j.apenergy.2022.119123_b25 article-title: Automatic P2P energy trading model based on reinforcement learning using long short-term delayed reward publication-title: Energies doi: 10.3390/en13205359 – year: 2017 ident: 10.1016/j.apenergy.2022.119123_b32 – volume: 113 start-page: 1100 year: 2014 ident: 10.1016/j.apenergy.2022.119123_b36 article-title: A practical battery wear model for electric vehicle charging applications publication-title: Appl Energy doi: 10.1016/j.apenergy.2013.08.062 – year: 2013 ident: 10.1016/j.apenergy.2022.119123_b2 – volume: 226 start-page: 261 year: 2018 ident: 10.1016/j.apenergy.2022.119123_b8 article-title: Peer-to-peer energy sharing through a two-stage aggregated battery control in a community microgrid publication-title: Appl Energy doi: 10.1016/j.apenergy.2018.05.097 – ident: 10.1016/j.apenergy.2022.119123_b30 doi: 10.24963/ijcai.2021/401 – start-page: 1 year: 2012 ident: 10.1016/j.apenergy.2022.119123_b40 article-title: Optimal inverter VAR control in distribution systems with high PV penetration – volume: 10 start-page: 5163 issue: 5 year: 2018 ident: 10.1016/j.apenergy.2022.119123_b17 article-title: Decentralized P2P energy trading under network constraints in a low-voltage network publication-title: IEEE Trans Smart Grid doi: 10.1109/TSG.2018.2878445 – volume: 11 start-page: 3095 issue: 4 year: 2019 ident: 10.1016/j.apenergy.2022.119123_b18 article-title: Integrating P2P energy trading with probabilistic distribution locational marginal pricing publication-title: IEEE Trans Smart Grid doi: 10.1109/TSG.2019.2963238 – start-page: 1 year: 2019 ident: 10.1016/j.apenergy.2022.119123_b24 article-title: Realistic peer-to-peer energy trading model for microgrids using deep reinforcement learning – volume: 36 start-page: 823 issue: 5 year: 1930 ident: 10.1016/j.apenergy.2022.119123_b44 article-title: On the theory of the Brownian motion publication-title: Phys Rev doi: 10.1103/PhysRev.36.823 – volume: 11 start-page: 3185 issue: 4 year: 2020 ident: 10.1016/j.apenergy.2022.119123_b5 article-title: Peer-to-peer trading in electricity networks: An overview publication-title: IEEE Trans Smart Grid doi: 10.1109/TSG.2020.2969657 – volume: 3 start-page: 94 issue: 2 year: 2018 ident: 10.1016/j.apenergy.2022.119123_b4 article-title: Using peer-to-peer energy-trading platforms to incentivize prosumers to form federated power plants publication-title: Nat Energy doi: 10.1038/s41560-017-0075-y – year: 2021 ident: 10.1016/j.apenergy.2022.119123_b29 article-title: Peer-to-peer energy trading and energy conversion in interconnected multi-energy microgrids using multi-agent deep reinforcement learning publication-title: IEEE Trans Smart Grid – year: 2020 ident: 10.1016/j.apenergy.2022.119123_b11 article-title: Peer-to-peer electricity transaction decision of user-side smart energy system based on SARSA reinforcement learning method publication-title: CSEE J Power Energy Syst – volume: 518 start-page: 529 issue: 7540 year: 2015 ident: 10.1016/j.apenergy.2022.119123_b22 article-title: Human-level control through deep reinforcement learning publication-title: Nature doi: 10.1038/nature14236 – volume: 11 start-page: 4513 issue: 5 year: 2020 ident: 10.1016/j.apenergy.2022.119123_b26 article-title: Deep reinforcement learning-based energy storage arbitrage with accurate lithium-ion battery degradation model publication-title: IEEE Trans Smart Grid doi: 10.1109/TSG.2020.2986333 – volume: 32 start-page: 3569 issue: 5 year: 2017 ident: 10.1016/j.apenergy.2022.119123_b37 article-title: Energy-sharing model with price-based demand response for microgrids of peer-to-peer prosumers publication-title: IEEE Trans Power Syst doi: 10.1109/TPWRS.2017.2649558 – volume: 33 start-page: 4061 issue: 4 year: 2017 ident: 10.1016/j.apenergy.2022.119123_b39 article-title: Distribution locational marginal pricing (DLMP) for congestion management and voltage support publication-title: IEEE Trans Power Syst doi: 10.1109/TPWRS.2017.2767632 – year: 2020 ident: 10.1016/j.apenergy.2022.119123_b27 – year: 2019 ident: 10.1016/j.apenergy.2022.119123_b45 – year: 2016 ident: 10.1016/j.apenergy.2022.119123_b46 – start-page: 38 year: 2020 ident: 10.1016/j.apenergy.2022.119123_b14 article-title: Real-time energy management of microgrid using reinforcement learning – start-page: 387 year: 2014 ident: 10.1016/j.apenergy.2022.119123_b42 article-title: Deterministic policy gradient algorithms – volume: 6 start-page: 739 issue: 7 year: 2020 ident: 10.1016/j.apenergy.2022.119123_b1 article-title: State-of-the-art analysis and perspectives for peer-to-peer energy trading publication-title: Engineering doi: 10.1016/j.eng.2020.06.002 – year: 2020 ident: 10.1016/j.apenergy.2022.119123_b15 – volume: 11 start-page: 1560 issue: 6 year: 2018 ident: 10.1016/j.apenergy.2022.119123_b10 article-title: Peer to peer distributed energy trading in smart grids: A survey publication-title: Energies doi: 10.3390/en11061560 – volume: 22 start-page: 439 issue: 2 year: 2007 ident: 10.1016/j.apenergy.2022.119123_b35 article-title: Comparison of photovoltaic array maximum power point tracking techniques publication-title: IEEE Trans Energy Convers doi: 10.1109/TEC.2006.874230 – volume: 34 start-page: 4005 issue: 5 year: 2018 ident: 10.1016/j.apenergy.2022.119123_b20 article-title: Multiclass energy management for peer-to-peer energy trading driven by prosumer preferences publication-title: IEEE Trans Power Syst doi: 10.1109/TPWRS.2018.2834472 – year: 2015 ident: 10.1016/j.apenergy.2022.119123_b43 – volume: 26 year: 2019 ident: 10.1016/j.apenergy.2022.119123_b7 article-title: Towards transactive energy systems: An analysis on current trends publication-title: Energy Strategy Rev doi: 10.1016/j.esr.2019.100418 – volume: 35 start-page: 2716 issue: 4 year: 2019 ident: 10.1016/j.apenergy.2022.119123_b19 article-title: A P2P-dominant distribution system architecture publication-title: IEEE Trans Power Syst doi: 10.1109/TPWRS.2019.2961330 – volume: 28 start-page: 2554 issue: 3 year: 2013 ident: 10.1016/j.apenergy.2022.119123_b38 article-title: Branch flow model: Relaxations and convexification—Part I publication-title: IEEE Trans Power Syst doi: 10.1109/TPWRS.2013.2255317 – volume: 9 start-page: 2647 issue: 4 year: 2016 ident: 10.1016/j.apenergy.2022.119123_b9 article-title: Incentivizing energy trading for interconnected microgrids publication-title: IEEE Trans Smart Grid doi: 10.1109/TSG.2016.2614988 – volume: 12 start-page: 5185 issue: 6 year: 2021 ident: 10.1016/j.apenergy.2022.119123_b31 article-title: A scalable privacy-preserving multi-agent deep reinforcement learning approach for large-scale peer-to-peer transactive energy trading publication-title: IEEE Trans Smart Grid doi: 10.1109/TSG.2021.3103917 – start-page: 157 year: 1994 ident: 10.1016/j.apenergy.2022.119123_b41 article-title: Markov games as a framework for multi-agent reinforcement learning – volume: 6 start-page: 62806 year: 2018 ident: 10.1016/j.apenergy.2022.119123_b13 article-title: Local energy trading behavior modeling with deep reinforcement learning publication-title: IEEE Access doi: 10.1109/ACCESS.2018.2876652 – volume: 17 start-page: 1111 issue: 2 year: 2020 ident: 10.1016/j.apenergy.2022.119123_b21 article-title: Decentralized local energy trading in microgrids with voltage management publication-title: IEEE Trans Ind Inf doi: 10.1109/TII.2020.2980160 – volume: 9 start-page: 4872 issue: 5 year: 2017 ident: 10.1016/j.apenergy.2022.119123_b33 article-title: Analysis of distribution locational marginal prices publication-title: IEEE Trans Smart Grid doi: 10.1109/TSG.2017.2673860
SSID	ssj0002120
Score	2.6017318
Snippet	In this paper, we investigate an energy cost minimization problem for prosumers participating in peer-to-peer energy trading. Due to (i) uncertainties caused...
SourceID	proquest crossref elsevier
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	119123
SubjectTerms	algorithms data collection Deep deterministic policy gradient energy energy costs issues and policy Markov decision process Multi-agent Peer-to-peer energy trading renewable energy sources Renewable generation
Title	Multi-agent deep deterministic policy gradient algorithm for peer-to-peer energy trading considering distribution network constraints
URI	https://dx.doi.org/10.1016/j.apenergy.2022.119123 https://www.proquest.com/docview/2661012784
Volume	317
WOSCitedRecordID	wos000799979200006&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 customDbUrl: eissn: 1872-9118 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0002120 issn: 0306-2619 databaseCode: AIEXJ dateStart: 19950101 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1Nb9MwGLZg4wAHBIOJ8SUjIS6Vi-MkTXycplaApsKhkyoulu3Y66YtCWmKxg_gf-OvpAENbRy4pFFkO26fp69fv34_AHibp1hqhROUUk1QkhGJOIkEElLjQmqNicSu2EQ2n-fLJf0S3MbWrpxAVpb51RWt_yvU5pkB24bO_gPc_aDmgbk3oJurgd1cbwW8C6lF3IZMjQqlanPxHi8uJbOryiB_jE4b5-tlDb6nVXPWri6dw2GtVIPaCtnPkfJxgW3j_Oytg7qr7WnvC5tvN5TKGpXeldw1cCUnfHaoPrltUHT9cL1Jh19a67sz1Kp6xXvd_oj7w6BNT9uZN9J-XVVhmQ1WCrL1aO2js_AE2d3aUPIa3WVUj22KORKja-W5Ny2cj3nt5zi2Y4ce2xWsO7Wff2azk-NjtpguF-_qb8jWFrNn8KHQyl2wS7KUGtm3e_hxuvzUr9gkpO_s5jiIJL_-1X9TYv5Yzp2OsngEHobNBTz0pHgM7qhyDzwYpJzcA_vTbWSjaRpE-_oJ-DngDbS8gb_xBnrewI43sOcNNLyBQ95A_01g4A0c8AYOeQMDb-CAN0_ByWy6OPqAQo0OJGMatyiOzYZaiklRiEkeC1ykKhWR4FRFWhRUcU6oVJgUiipKeBQpUWgqico4T2icxPtgp6xK9QxAHWNlkzelWGsjNCZc4oznIk6FTqlQ0QFIux-dyZDA3k7ugnWeiuesA4tZsJgH6wC87_vVPoXLjT1ohykLiqhXMJnh5Y1933QkYEZS2-M3Xqpqs2ZWFbaOHnny_BZtXoD72__RS7DTNhv1CtyT39uzdfM6MPgXKZXAog
linkProvider	Elsevier
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Multi-agent+deep+deterministic+policy+gradient+algorithm+for+peer-to-peer+energy+trading+considering+distribution+network+constraints&rft.jtitle=Applied+energy&rft.au=Samende%2C+Cephas&rft.au=Cao%2C+Jun&rft.au=Fan%2C+Zhong&rft.date=2022-07-01&rft.issn=0306-2619&rft.volume=317+p.119123-&rft_id=info:doi/10.1016%2Fj.apenergy.2022.119123&rft.externalDBID=NO_FULL_TEXT
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0306-2619&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0306-2619&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0306-2619&client=summon