Distributed Reinforcement Learning Algorithm for Dynamic Economic Dispatch With Unknown Generation Cost Functions

In this article, the dynamic economic dispatch (DED) problem for smart grid is solved under the assumption that no knowledge of the mathematical formulation of the actual generation cost functions is available. The objective of the DED problem is to find the optimal power output of each unit at each...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	IEEE transactions on industrial informatics Jg. 16; H. 4; S. 2258 - 2267
Hauptverfasser:	Dai, Pengcheng, Yu, Wenwu, Wen, Guanghui, Baldi, Simone
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	Piscataway IEEE 01.04.2020 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:	Algorithms Approximation algorithms Cost function Distributed reinforcement learning dynamic economic dispatch (DED) Economics Heuristic algorithms Machine learning Mathematical analysis multiplier splitting Optimization Power dispatch Reinforcement learning Smart grid Smart grids state-action-value function approximation
ISSN:	1551-3203, 1941-0050
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Abstract	In this article, the dynamic economic dispatch (DED) problem for smart grid is solved under the assumption that no knowledge of the mathematical formulation of the actual generation cost functions is available. The objective of the DED problem is to find the optimal power output of each unit at each time so as to minimize the total generation cost. To address the lack of a priori knowledge, a new distributed reinforcement learning optimization algorithm is proposed. The algorithm combines the state-action-value function approximation with a distributed optimization based on multiplier splitting. Theoretical analysis of the proposed algorithm is provided to prove the feasibility of the algorithm, and several case studies are presented to demonstrate its effectiveness.
AbstractList	In this article, the dynamic economic dispatch (DED) problem for smart grid is solved under the assumption that no knowledge of the mathematical formulation of the actual generation cost functions is available. The objective of the DED problem is to find the optimal power output of each unit at each time so as to minimize the total generation cost. To address the lack of a priori knowledge, a new distributed reinforcement learning optimization algorithm is proposed. The algorithm combines the state-action-value function approximation with a distributed optimization based on multiplier splitting. Theoretical analysis of the proposed algorithm is provided to prove the feasibility of the algorithm, and several case studies are presented to demonstrate its effectiveness.
Author	Baldi, Simone Yu, Wenwu Wen, Guanghui Dai, Pengcheng
Author_xml	– sequence: 1 givenname: Pengcheng surname: Dai fullname: Dai, Pengcheng email: jldaipc@163.com organization: School of Mathematics, Southeast University, Nanjing, China – sequence: 2 givenname: Wenwu orcidid: 0000-0003-3755-179X surname: Yu fullname: Yu, Wenwu email: wwyu@seu.edu.cn organization: School of Mathematics, Southeast University, Nanjing, China – sequence: 3 givenname: Guanghui orcidid: 0000-0003-0070-8597 surname: Wen fullname: Wen, Guanghui email: wenguanghui@gmail.com organization: School of Mathematics, Southeast University, Nanjing, China – sequence: 4 givenname: Simone orcidid: 0000-0001-9752-8925 surname: Baldi fullname: Baldi, Simone email: s.baldi@tudelft.nl organization: School of Mathematics, Southeast University, Nanjing, China
BookMark	eNp9UE1LAzEQDaKgVu-Cl4DnrZPMZrc5Slu1UBCkxeOSTbNtapvUJEX679214sGDp3nDvA_mXZJT550h5IZBnzGQ97PJpM-ByT6XiHmOJ-SCyZxlAAJOWywEy5ADnpPLGNcAWALKC_IxsjEFW--TWdBXY13jgzZb4xKdGhWcdUv6sFn6YNNqS9sjHR2c2lpNx9o734HWYaeSXtG3lkPn7t35T0efjDNBJesdHfqY6OPe6W6LV-SsUZtorn9mj8wfx7PhczZ9eZoMH6aZRsSUSaihUbUuuZYG5EJjwQuRM63Lpii1aMrFooS8ZkIKUEwJrg0UhWJachzUHHvk7ui7C_5jb2Kq1n4fXBtZccyF4IggWhYcWTr4GINpql2wWxUOFYOqK7Zqi626YqufYltJ8Ueibfr-NAVlN_8Jb49Ca4z5zRmUA4lS4hcni4iq
CODEN	ITIICH
CitedBy_id	crossref_primary_10_1109_TSG_2023_3339541 crossref_primary_10_1016_j_jclepro_2021_128318 crossref_primary_10_1016_j_enconman_2025_119827 crossref_primary_10_1109_TNNLS_2021_3054778 crossref_primary_10_1109_TII_2023_3296887 crossref_primary_10_1109_TNNLS_2023_3337387 crossref_primary_10_61435_ijred_2024_60119 crossref_primary_10_1109_TPWRS_2021_3070161 crossref_primary_10_1109_TSG_2023_3329459 crossref_primary_10_1002_int_22945 crossref_primary_10_1109_JIOT_2024_3377201 crossref_primary_10_1007_s00202_023_01947_8 crossref_primary_10_1109_TNSE_2025_3527466 crossref_primary_10_1109_JIOT_2021_3067951 crossref_primary_10_3390_robotics11020035 crossref_primary_10_1109_TII_2024_3465601 crossref_primary_10_1016_j_ijepes_2023_109195 crossref_primary_10_1109_TSG_2023_3331467 crossref_primary_10_1007_s10489_023_04489_5 crossref_primary_10_1016_j_energy_2022_125624 crossref_primary_10_1016_j_epsr_2022_108828 crossref_primary_10_1109_TII_2022_3151772 crossref_primary_10_1109_TSTE_2024_3355123 crossref_primary_10_3390_pr12061199 crossref_primary_10_1109_TSMC_2024_3462762 crossref_primary_10_1016_j_sysconle_2023_105657 crossref_primary_10_1016_j_apenergy_2023_121704 crossref_primary_10_3390_sym16030322 crossref_primary_10_1109_TPWRS_2023_3298007 crossref_primary_10_1109_TCNS_2024_3510598 crossref_primary_10_1016_j_ijepes_2020_106759 crossref_primary_10_1016_j_ijepes_2024_109867 crossref_primary_10_1109_TETCI_2024_3360305 crossref_primary_10_1016_j_neucom_2021_08_140 crossref_primary_10_1016_j_neucom_2023_01_038 crossref_primary_10_1016_j_apenergy_2023_122121 crossref_primary_10_1109_TNNLS_2023_3234049 crossref_primary_10_1016_j_ifacol_2022_05_043 crossref_primary_10_1109_ACCESS_2021_3125102 crossref_primary_10_1177_01423312221110999 crossref_primary_10_1049_gtd2_12206 crossref_primary_10_1109_TPWRS_2022_3217905 crossref_primary_10_1016_j_asoc_2020_106882 crossref_primary_10_1109_TII_2021_3078110 crossref_primary_10_1109_JSYST_2023_3315833 crossref_primary_10_1109_TNSE_2020_3018871 crossref_primary_10_3390_pr11051513 crossref_primary_10_1109_TCYB_2021_3082639 crossref_primary_10_1109_TVT_2024_3434969 crossref_primary_10_3390_en17030550 crossref_primary_10_1080_00207721_2025_2461016 crossref_primary_10_3390_app15020900 crossref_primary_10_1002_acs_3234 crossref_primary_10_1016_j_isatra_2025_01_009 crossref_primary_10_1109_JIOT_2023_3284510 crossref_primary_10_1109_TETCI_2023_3299294 crossref_primary_10_1109_TPWRS_2022_3159825 crossref_primary_10_1038_s41598_025_00492_7 crossref_primary_10_1631_FITEE_2000205 crossref_primary_10_1007_s00521_022_08089_1 crossref_primary_10_1109_TII_2020_3016336 crossref_primary_10_1109_TNSE_2024_3384505 crossref_primary_10_1109_TNNLS_2022_3160645 crossref_primary_10_1109_JAS_2023_123402 crossref_primary_10_1109_TNNLS_2021_3139138 crossref_primary_10_1109_TIA_2025_3529675
Cites_doi	10.1109/TCST.2018.2816902 10.1007/s11432-016-9114-y 10.1109/TAC.2010.2041686 10.1016/j.sysconle.2015.06.006 10.1109/TAC.2004.834113 10.1038/nature14236 10.1109/TSMCB.2008.920269 10.1109/TCNS.2015.2399191 10.1515/9781400873173 10.1109/TII.2013.2287807 10.1109/TII.2015.2479558 10.1109/TSG.2016.2623980 10.1109/SURV.2011.101911.00087 10.1016/j.neucom.2013.06.037 10.1109/TNN.1998.712192 10.1109/MPE.2009.934876 10.1049/iet-gtd.2015.1345 10.1109/TPWRS.2018.2889989 10.1002/rnc.1760 10.1109/TIE.2016.2617832 10.1109/TPWRS.2013.2271640 10.1109/TII.2017.2772088 10.1109/TSG.2015.2434831 10.1016/j.rser.2013.10.022 10.1109/TIE.2016.2542134 10.1109/PSCE.2009.4840087 10.1109/TAC.2008.2009515 10.1109/TIE.2016.2615037 10.1109/TII.2011.2166794 10.1109/TSG.2018.2834368 10.1016/j.automatica.2016.08.007 10.1109/TII.2012.2209668
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2020
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2020
DBID	97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
DOI	10.1109/TII.2019.2933443
DatabaseName	IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional
DatabaseTitleList	Technology Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Economics Engineering
EISSN	1941-0050
EndPage	2267
ExternalDocumentID	10_1109_TII_2019_2933443 8789399
Genre	orig-research
GrantInformation_xml	– fundername: Six Talent Peaks of Jiangsu Province grantid: 2019-DZXX-006 – fundername: National Ten Thousand Talent Program for Young Top-Notch Talents grantid: W2070082 – fundername: Fundamental Research Funds for the Central Universities grantid: 4007019109 funderid: 10.13039/501100012226 – fundername: National Natural Science Foundation of China grantid: 61673107; 61673104 funderid: 10.13039/501100001809 – fundername: Ministry of Education grantid: 6141A020223 funderid: 10.13039/100010449 – fundername: Jiangsu Provincial Key Laboratory of Networked Collective Intelligence grantid: BM2017002 – fundername: Special Guiding Funds for Double First-Class grantid: 4007019201
GroupedDBID	0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFS ACIWK AENEX AETIX AGQYO AGSQL AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD HZ~ IFIPE IPLJI JAVBF LAI M43 O9- OCL P2P RIA RIE RNS AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
ID	FETCH-LOGICAL-c333t-90b0fabc72c9e09dc3626541cc7f67c5f7dd704b15950a1a52ce066a1c9238b23
IEDL.DBID	RIE
ISICitedReferencesCount	89
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000510901000009&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	1551-3203
IngestDate	Mon Jun 30 10:22:21 EDT 2025 Sat Nov 29 04:16:48 EST 2025 Tue Nov 18 22:35:39 EST 2025 Wed Aug 27 02:38:57 EDT 2025
IsDoiOpenAccess	false
IsOpenAccess	true
IsPeerReviewed	false
IsScholarly	true
Issue	4
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c333t-90b0fabc72c9e09dc3626541cc7f67c5f7dd704b15950a1a52ce066a1c9238b23
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0001-9752-8925 0000-0003-0070-8597 0000-0003-3755-179X
OpenAccessLink	http://resolver.tudelft.nl/uuid:49d9f5f6-8fbc-419f-a599-3368d6cecf0d
PQID	2345523305
PQPubID	85507
PageCount	10
ParticipantIDs	crossref_primary_10_1109_TII_2019_2933443 crossref_citationtrail_10_1109_TII_2019_2933443 proquest_journals_2345523305 ieee_primary_8789399
PublicationCentury	2000
PublicationDate	2020-04-01
PublicationDateYYYYMMDD	2020-04-01
PublicationDate_xml	– month: 04 year: 2020 text: 2020-04-01 day: 01
PublicationDecade	2020
PublicationPlace	Piscataway
PublicationPlace_xml	– name: Piscataway
PublicationTitle	IEEE transactions on industrial informatics
PublicationTitleAbbrev	TII
PublicationYear	2020
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 ref34 ref12 ref15 ref14 ref11 ref32 ref10 ref2 ref1 ref17 ref16 ref19 ref18 ref24 ref23 ref26 ref25 ref20 ref22 ref21 wood (ref33) 2012 mnih (ref31) 2015; 518 ref28 ref27 ref29 ref8 ref7 ref9 ref4 ref3 mnih (ref30) 2013 ref6 ref5
References_xml	– ident: ref21 doi: 10.1109/TCST.2018.2816902 – year: 2012 ident: ref33 publication-title: Power Generation Operation and Control – ident: ref19 doi: 10.1007/s11432-016-9114-y – ident: ref9 doi: 10.1109/TAC.2010.2041686 – ident: ref17 doi: 10.1016/j.sysconle.2015.06.006 – ident: ref7 doi: 10.1109/TAC.2004.834113 – volume: 518 start-page: 529 year: 2015 ident: ref31 article-title: Human-level control through deep reinforcement learning publication-title: Nature doi: 10.1038/nature14236 – ident: ref26 doi: 10.1109/TSMCB.2008.920269 – ident: ref13 doi: 10.1109/TCNS.2015.2399191 – ident: ref32 doi: 10.1515/9781400873173 – ident: ref16 doi: 10.1109/TII.2013.2287807 – ident: ref20 doi: 10.1109/TII.2015.2479558 – ident: ref23 doi: 10.1109/TSG.2016.2623980 – ident: ref1 doi: 10.1109/SURV.2011.101911.00087 – ident: ref28 doi: 10.1016/j.neucom.2013.06.037 – ident: ref25 doi: 10.1109/TNN.1998.712192 – ident: ref2 doi: 10.1109/MPE.2009.934876 – ident: ref22 doi: 10.1049/iet-gtd.2015.1345 – ident: ref34 doi: 10.1109/TPWRS.2018.2889989 – ident: ref29 doi: 10.1002/rnc.1760 – ident: ref12 doi: 10.1109/TIE.2016.2617832 – ident: ref10 doi: 10.1109/TPWRS.2013.2271640 – ident: ref18 doi: 10.1109/TII.2017.2772088 – ident: ref15 doi: 10.1109/TSG.2015.2434831 – ident: ref3 doi: 10.1016/j.rser.2013.10.022 – ident: ref27 doi: 10.1109/TIE.2016.2542134 – ident: ref5 doi: 10.1109/PSCE.2009.4840087 – ident: ref8 doi: 10.1109/TAC.2008.2009515 – year: 2013 ident: ref30 article-title: Playing Atari with deep reinforcement learning publication-title: arXiv 1312 5602 – ident: ref11 doi: 10.1109/TIE.2016.2615037 – ident: ref4 doi: 10.1109/TII.2011.2166794 – ident: ref24 doi: 10.1109/TSG.2018.2834368 – ident: ref14 doi: 10.1016/j.automatica.2016.08.007 – ident: ref6 doi: 10.1109/TII.2012.2209668
SSID	ssj0037039
Score	2.5687609
Snippet	In this article, the dynamic economic dispatch (DED) problem for smart grid is solved under the assumption that no knowledge of the mathematical formulation of...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	2258
SubjectTerms	Algorithms Approximation algorithms Cost function Distributed reinforcement learning dynamic economic dispatch (DED) Economics Heuristic algorithms Machine learning Mathematical analysis multiplier splitting Optimization Power dispatch Reinforcement learning Smart grid Smart grids state-action-value function approximation
Title	Distributed Reinforcement Learning Algorithm for Dynamic Economic Dispatch With Unknown Generation Cost Functions
URI	https://ieeexplore.ieee.org/document/8789399 https://www.proquest.com/docview/2345523305
Volume	16
WOSCitedRecordID	wos000510901000009&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1941-0050 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0037039 issn: 1551-3203 databaseCode: RIE dateStart: 20050101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3dS-QwEB9UBO8e1NMT9_wgD_ciWDdtmk3zKOqiICKi6FtJp4kraFd36_39N0nbRVEE3wpNQulvMl_J_AbgrzNZ6mKJkaFYIUqdVJHhaRG5gRhYw2MsRSgUPlcXF9ndnb6cg_1ZLYy1Nlw-swf-MZzll2N89amyfqbIumo9D_NKDZparU7rCpJcHbhRZRyJhIvuSJLr_vXZmb_DpQ_ItIk0Fe9MUOip8kERB-syXPned63CcutFssMG9l8wZ6s1WOqKjKdr8PMNz-A6vBx7elzf2cqW7MoGslQMeUHW8qves8PH-_HkoR49MXrJjptG9axbktEKz6S1R-yWxrCbyufiKtaQVnts2dF4WrMhWckgyL_hZnhyfXQatb0WIhRC1JHmBXemQJWgtlyX6GlqZBojKjdQKJ0qS0UgkvcjuYmNTNCSt2JiJA8xKxKxAQvVuLKbwHjpLEoCWQh_cixN6rSUmElXFDyzqgf97vfn2BKR-34Yj3kISLjOCbDcA5a3gPVgbzbjuSHh-GLsugdoNq7FpgfbHcJ5u0uneSJSSYE4qbw_n8_agh-Jj6_DTZ1tWKgnr3YHFvFf_TCd7AYB_A8lYtoH
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3db9MwED-NgTR44GMDrTDAD7wgkdWJ7Tp-nLZVqygVQp3YW-Rc7G3SSLc24-_n7CTVpiEk3iLFtqL8zvdl3-8APnmbS58qTCzFCon0SieWyzLxIzFylqdYiVgoPNWzWX52Zr5vwJd1LYxzLl4-c_vhMZ7lVwu8DamyYa7JuhrzCB4rKTPeVmv1eleQ7JrIjqrSRGRc9IeS3Aznk0m4xWX2ybgJKcU9IxS7qjxQxdG-jF_835e9hOedH8kOWuBfwYart2GrLzNebcOzO0yDO3BzFAhyQ28rV7EfLtKlYswMso5h9ZwdXJ0vlpfNxS9GL9lR26qe9UsyWuGa9PYF-0lj2GkdsnE1a2mrA7rscLFq2JjsZBTl13A6Pp4fniRdt4UEhRBNYnjJvS1RZ2gcNxUGoholU0TtRxqV11WlCUbyfxS3qVUZOvJXbIrkI-ZlJt7AZr2o3S4wXnmHimAWIpwdKyu9UQpz5cuS504PYNj__gI7KvLQEeOqiCEJNwUBVgTAig6wAXxez7huaTj-MXYnALQe12EzgL0e4aLbp6siE1JRKE5K7-3fZ32ErZP5t2kxncy-voOnWYi2472dPdhslrfuPTzB383lavkhCuMfooTdTg
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Distributed+Reinforcement+Learning+Algorithm+for+Dynamic+Economic+Dispatch+With+Unknown+Generation+Cost+Functions&rft.jtitle=IEEE+transactions+on+industrial+informatics&rft.au=Dai%2C+Pengcheng&rft.au=Yu%2C+Wenwu&rft.au=Wen%2C+Guanghui&rft.au=Baldi%2C+Simone&rft.date=2020-04-01&rft.pub=IEEE&rft.issn=1551-3203&rft.volume=16&rft.issue=4&rft.spage=2258&rft.epage=2267&rft_id=info:doi/10.1109%2FTII.2019.2933443&rft.externalDocID=8789399
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1551-3203&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1551-3203&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1551-3203&client=summon