A Fast Technique for Smart Home Management: ADP With Temporal Difference Learning
This paper presents a computationally efficient smart home energy management system (SHEMS) using an approximate dynamic programming (ADP) approach with temporal difference learning for scheduling distributed energy resources. This approach improves the performance of an SHEMS by incorporating stoch...
Gespeichert in:
| Veröffentlicht in: | IEEE transactions on smart grid Jg. 9; H. 4; S. 3291 - 3303 |
|---|---|
| Hauptverfasser: | , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
IEEE
01.07.2018
|
| Schlagworte: | |
| ISSN: | 1949-3053, 1949-3061 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | This paper presents a computationally efficient smart home energy management system (SHEMS) using an approximate dynamic programming (ADP) approach with temporal difference learning for scheduling distributed energy resources. This approach improves the performance of an SHEMS by incorporating stochastic energy consumption and PV generation models over a horizon of several days, using only the computational power of existing smart meters. In this paper, we consider a PV-storage (thermal and battery) system, however, our method can extend to multiple controllable devices without the exponential growth in computation that other methods such as dynamic programming (DP) and stochastic mixed-integer linear programming (MILP) suffer from. Specifically, probability distributions associated with the PV output and demand are kernel estimated from empirical data collected during the Smart Grid Smart City project in NSW, Australia. Our results show that ADP computes a solution much faster than both DP and stochastic MILP, and provides only a slight reduction in quality compared to the optimal DP solution. In addition, incorporating a thermal energy storage unit using the proposed ADP-based SHEMS reduces the daily electricity cost by up to 26.3% without a noticeable increase in the computational burden. Moreover, ADP with a two-day decision horizon reduces the average yearly electricity cost by a 4.6% over a daily DP method, yet requires less than half of the computational effort. |
|---|---|
| AbstractList | This paper presents a computationally efficient smart home energy management system (SHEMS) using an approximate dynamic programming (ADP) approach with temporal difference learning for scheduling distributed energy resources. This approach improves the performance of an SHEMS by incorporating stochastic energy consumption and PV generation models over a horizon of several days, using only the computational power of existing smart meters. In this paper, we consider a PV-storage (thermal and battery) system, however, our method can extend to multiple controllable devices without the exponential growth in computation that other methods such as dynamic programming (DP) and stochastic mixed-integer linear programming (MILP) suffer from. Specifically, probability distributions associated with the PV output and demand are kernel estimated from empirical data collected during the Smart Grid Smart City project in NSW, Australia. Our results show that ADP computes a solution much faster than both DP and stochastic MILP, and provides only a slight reduction in quality compared to the optimal DP solution. In addition, incorporating a thermal energy storage unit using the proposed ADP-based SHEMS reduces the daily electricity cost by up to 26.3% without a noticeable increase in the computational burden. Moreover, ADP with a two-day decision horizon reduces the average yearly electricity cost by a 4.6% over a daily DP method, yet requires less than half of the computational effort. |
| Author | Chapman, Archie C. Keerthisinghe, Chanaka Verbic, Gregor |
| Author_xml | – sequence: 1 givenname: Chanaka orcidid: 0000-0003-2803-5224 surname: Keerthisinghe fullname: Keerthisinghe, Chanaka email: chanaka.keerthisinghe@sydney.edu.au organization: School of Electrical and Information Engineering, University of Sydney, Sydney, NSW, Australia – sequence: 2 givenname: Gregor orcidid: 0000-0003-4949-768X surname: Verbic fullname: Verbic, Gregor email: gregor.verbic@sydney.edu.au organization: School of Electrical and Information Engineering, University of Sydney, Sydney, NSW, Australia – sequence: 3 givenname: Archie C. orcidid: 0000-0002-5055-3004 surname: Chapman fullname: Chapman, Archie C. email: archie.chapman@sydney.edu.au organization: School of Electrical and Information Engineering, University of Sydney, Sydney, NSW, Australia |
| BookMark | eNp9kMFOAjEQhhuDiYjcTbz0BRbbnW6beiMgYIJRA8bjptudhRq2i9168O1dAuHgwbn8c_i_Sea7Jj3feCTklrMR50zfr1fzUcq4HKUy1UKxC9LnWugEmOS9857BFRm27SfrBgC6ap-8jenMtJGu0W69-_pGWjWBrmoTIl00NdJn480Ga_TxgY6nr_TDxW3XrvdNMDs6dVWFAb1FukQTvPObG3JZmV2Lw1MOyPvscT1ZJMuX-dNkvExsKiEmRmlIrWBaGmkVh0zIkjHkUhepwFIJzgQziAAKM1sBFEqWhVVpYQpeqgIGRB7v2tC0bcAqty6a6Bofg3G7nLP84Cbv3OQHN_nJTQeyP-A-uO7hn_-QuyPiEPFcV0pkGhj8ArUJb8c |
| CODEN | ITSGBQ |
| CitedBy_id | crossref_primary_10_1109_TII_2018_2839059 crossref_primary_10_1049_iet_gtd_2019_1744 crossref_primary_10_1016_j_epsr_2022_108120 crossref_primary_10_1016_j_ijepes_2022_108359 crossref_primary_10_1109_TII_2021_3095141 crossref_primary_10_1109_TSG_2025_3547985 crossref_primary_10_1109_TSG_2019_2956785 crossref_primary_10_1109_JSYST_2020_3033128 crossref_primary_10_1016_j_ijepes_2019_105542 crossref_primary_10_1109_JIOT_2022_3152586 crossref_primary_10_1002_acs_3282 crossref_primary_10_1007_s10462_022_10261_x crossref_primary_10_1109_ACCESS_2024_3471076 crossref_primary_10_1109_TSTE_2019_2951616 crossref_primary_10_1016_j_epsr_2022_108119 crossref_primary_10_3390_en11123494 crossref_primary_10_1016_j_ijepes_2021_107413 crossref_primary_10_3390_en13102562 crossref_primary_10_1109_TIA_2020_2981916 crossref_primary_10_3390_en16083517 crossref_primary_10_1109_TSTE_2024_3361674 crossref_primary_10_1007_s00202_025_02958_3 crossref_primary_10_1016_j_jobe_2024_110206 crossref_primary_10_1109_TIA_2023_3348768 crossref_primary_10_1049_iet_rpg_2019_0536 crossref_primary_10_1002_jnm_3015 crossref_primary_10_1016_j_compbiomed_2022_105458 crossref_primary_10_1016_j_enconman_2021_114322 crossref_primary_10_1016_j_rser_2020_110000 crossref_primary_10_1177_0957650920942998 crossref_primary_10_3390_en12081501 crossref_primary_10_1109_TSTE_2018_2855039 crossref_primary_10_1016_j_scs_2021_102969 crossref_primary_10_3390_pr13072148 crossref_primary_10_1016_j_ifacsc_2020_100106 crossref_primary_10_1109_TSTE_2020_3021226 crossref_primary_10_1109_TPWRS_2022_3227345 crossref_primary_10_1016_j_apenergy_2019_113669 crossref_primary_10_1109_TSG_2022_3225805 crossref_primary_10_1109_ACCESS_2020_3024901 crossref_primary_10_1016_j_seta_2020_100834 crossref_primary_10_1109_TSG_2018_2878445 crossref_primary_10_1109_TSTE_2022_3153609 crossref_primary_10_1109_TSG_2020_2978061 crossref_primary_10_1016_j_segan_2023_101133 crossref_primary_10_1109_TCST_2022_3208822 crossref_primary_10_1109_TIE_2018_2850023 crossref_primary_10_1016_j_asoc_2020_106882 crossref_primary_10_1109_TSG_2018_2798039 crossref_primary_10_1016_j_energy_2023_129593 crossref_primary_10_1016_j_renene_2022_04_085 crossref_primary_10_1109_TSG_2021_3111029 crossref_primary_10_1109_TCST_2021_3132662 crossref_primary_10_1016_j_compind_2023_103974 crossref_primary_10_1109_ACCESS_2020_3023665 crossref_primary_10_1109_TSTE_2019_2927237 crossref_primary_10_1109_ACCESS_2021_3071993 crossref_primary_10_3390_en16093897 crossref_primary_10_1109_TSG_2020_2971427 crossref_primary_10_1016_j_energy_2025_135386 crossref_primary_10_1002_ecj_12452 crossref_primary_10_1016_j_neucom_2023_127185 crossref_primary_10_1109_TSTE_2021_3086846 crossref_primary_10_1016_j_epsr_2020_106543 crossref_primary_10_3390_app11010158 |
| Cites_doi | 10.1109/TSG.2012.2212729 10.1109/IGCC.2014.7039146 10.1002/9780470182963 10.1109/AUPEC.2014.6966552 10.1109/TSG.2013.2293131 10.1109/CDC.2011.6161081 10.1109/TII.2012.2230637 10.1109/ISGT-Asia.2011.6167090 10.1109/ISGT.2011.5759191 10.1109/PSCC.2016.7540924 10.1287/ijoc.1110.0470 10.1109/FSKD.2011.6019758 10.1287/mnsc.47.8.1101.10231 10.1109/TSG.2012.2201182 10.1109/TSG.2010.2053053 10.1109/ISGTEurope.2013.6695439 10.1109/PSCC.2014.7038377 10.1109/SmartGridComm.2011.6102379 10.1109/TSG.2013.2279110 10.1162/neco.1994.6.6.1185 10.1109/ADPRL.2014.7010626 10.1109/TPWRS.2015.2414880 10.1109/TSG.2016.2516559 10.1109/TSG.2012.2212032 10.1109/ISGT.2011.5759152 10.1109/JPROC.2011.2109671 10.1109/ISGTEurope.2013.6695463 |
| ContentType | Journal Article |
| DBID | 97E RIA RIE AAYXX CITATION |
| DOI | 10.1109/TSG.2016.2629470 |
| DatabaseName | IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1949-3061 |
| EndPage | 3303 |
| ExternalDocumentID | 10_1109_TSG_2016_2629470 7745930 |
| Genre | orig-research |
| GroupedDBID | 0R~ 4.4 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACIWK AENEX AGQYO AGSQL AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD HZ~ IFIPE IPLJI JAVBF M43 O9- OCL P2P RIA RIE RNS AAYXX CITATION |
| ID | FETCH-LOGICAL-c263t-a7932c4096a6c713546d00e169b24ed741040aee337e5cf33b76dbc72bab1d7b3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 91 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000443196400081&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1949-3053 |
| IngestDate | Tue Nov 18 21:49:35 EST 2025 Sat Nov 29 03:45:51 EST 2025 Wed Aug 27 02:50:28 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 4 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c263t-a7932c4096a6c713546d00e169b24ed741040aee337e5cf33b76dbc72bab1d7b3 |
| ORCID | 0000-0003-4949-768X 0000-0002-5055-3004 0000-0003-2803-5224 |
| PageCount | 13 |
| ParticipantIDs | crossref_primary_10_1109_TSG_2016_2629470 ieee_primary_7745930 crossref_citationtrail_10_1109_TSG_2016_2629470 |
| PublicationCentury | 2000 |
| PublicationDate | 2018-July 2018-7-00 |
| PublicationDateYYYYMMDD | 2018-07-01 |
| PublicationDate_xml | – month: 07 year: 2018 text: 2018-July |
| PublicationDecade | 2010 |
| PublicationTitle | IEEE transactions on smart grid |
| PublicationTitleAbbrev | TSG |
| PublicationYear | 2018 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| References | ref13 ref37 ref15 ref36 ref14 ref31 ref33 ref32 ref2 ref39 ref38 ref16 ref19 ref18 (ref1) 2014 powell (ref30) 2005 (ref3) 0 (ref11) 2015 bertsekas (ref21) 2005; 1 salas (ref25) 2013 (ref9) 2015 wang (ref17) 2012 ref24 ref23 ref26 ref20 ref22 (ref35) 2015 (ref41) 2012 ref28 ref27 (ref10) 2014 ref29 ref8 ref7 (ref4) 0 ref6 ratnam (ref34) 2015 ref5 ref40 (ref12) 2014 |
| References_xml | – ident: ref13 doi: 10.1109/TSG.2012.2212729 – ident: ref40 doi: 10.1109/IGCC.2014.7039146 – ident: ref29 doi: 10.1002/9780470182963 – ident: ref14 doi: 10.1109/AUPEC.2014.6966552 – ident: ref33 doi: 10.1109/TSG.2013.2293131 – ident: ref18 doi: 10.1109/CDC.2011.6161081 – ident: ref16 doi: 10.1109/TII.2012.2230637 – year: 2012 ident: ref41 article-title: Water heating data collection and analysis – ident: ref22 doi: 10.1109/ISGT-Asia.2011.6167090 – ident: ref5 doi: 10.1109/ISGT.2011.5759191 – year: 2014 ident: ref1 article-title: Australian government bureau of resources and energy economics – ident: ref24 doi: 10.1109/PSCC.2016.7540924 – ident: ref28 doi: 10.1287/ijoc.1110.0470 – year: 2015 ident: ref35 publication-title: Smart-Grid Smart-City Customer Trial Data – year: 0 ident: ref4 publication-title: Flexible Pricing FAQs – year: 0 ident: ref3 publication-title: BGE Time of Use Pricing – volume: 1 year: 2005 ident: ref21 publication-title: Dynamic Programming and Optimal Control – year: 2014 ident: ref12 article-title: The economics of grid defection when and where distributed solar generation plus storage competes with traditional utility service – year: 2015 ident: ref9 publication-title: Australian PV Market Since April 2001 – ident: ref39 doi: 10.1109/FSKD.2011.6019758 – year: 2013 ident: ref25 publication-title: Benchmarking A Scalable Approximate Dynamic Programming Algorithm for Stochastic Control of Multidimensional Energy Storage Problems – ident: ref37 doi: 10.1287/mnsc.47.8.1101.10231 – start-page: 1 year: 2015 ident: ref34 article-title: Residential load and rooftop PV generation: An Australian distribution network dataset publication-title: Int J Sustain Energy – ident: ref6 doi: 10.1109/TSG.2012.2201182 – ident: ref20 doi: 10.1109/TSG.2010.2053053 – ident: ref32 doi: 10.1109/ISGTEurope.2013.6695439 – year: 2005 ident: ref30 publication-title: Approximate Dynamic Programming for Large-Scale Resource Allocation Problems – ident: ref23 doi: 10.1109/PSCC.2014.7038377 – ident: ref38 doi: 10.1109/SmartGridComm.2011.6102379 – start-page: 1 year: 2012 ident: ref17 article-title: Optimal dispatching model of smart home energy management system publication-title: Proc Innov Smart Grid Technol (ISGT-ASIA) – ident: ref7 doi: 10.1109/TSG.2013.2279110 – year: 2015 ident: ref11 article-title: Emerging technologies information – ident: ref36 doi: 10.1162/neco.1994.6.6.1185 – ident: ref26 doi: 10.1109/ADPRL.2014.7010626 – ident: ref8 doi: 10.1109/TPWRS.2015.2414880 – ident: ref2 doi: 10.1109/TSG.2016.2516559 – ident: ref15 doi: 10.1109/TSG.2012.2212032 – year: 2014 ident: ref10 article-title: The integrated grid realizing the full value of central and distributed energy resources – ident: ref19 doi: 10.1109/ISGT.2011.5759152 – ident: ref27 doi: 10.1109/JPROC.2011.2109671 – ident: ref31 doi: 10.1109/ISGTEurope.2013.6695463 |
| SSID | ssj0000333629 |
| Score | 2.5140598 |
| Snippet | This paper presents a computationally efficient smart home energy management system (SHEMS) using an approximate dynamic programming (ADP) approach with... |
| SourceID | crossref ieee |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 3291 |
| SubjectTerms | approximate dynamic programming Australia Batteries Demand response distributed energy resources Dynamic programming Energy management Optimization smart home energy management Smart homes stochastic mixed-integer linear programming Stochastic processes temporal difference learning value function approximation |
| Title | A Fast Technique for Smart Home Management: ADP With Temporal Difference Learning |
| URI | https://ieeexplore.ieee.org/document/7745930 |
| Volume | 9 |
| WOSCitedRecordID | wos000443196400081&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1949-3061 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000333629 issn: 1949-3053 databaseCode: RIE dateStart: 20100101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELZKxQADr4IoL3lgQSJtajt2whZRClNV1CK6RfEjUKm0qE35_ZwTN3RASCxRFJ2l6PPrznf-PoSuJSdSExZ4oQ6lx0xGvFQp38t8rjPKtcic2ITo98PxOBrU0G11F8YYUxSfmZZ9LXL5eq5W9qisDa5KEFEI0LeE4OVdreo8xacU1uKoSCIzm84P6Dor6Uft0fDRlnHxFgEbZpWJN3ahDVmVYlfp7f_vfw7QnvMecVx29yGqmdkR2t3gFGyg5xj30mWOR2tyVgxuKR5-wBDBVhMd_xS83OG4O8Cvk_wdrAuGqinuOsEUZbCjXn07Ri-9h9H9k-d0EzxFOM29FOYcURC48ZQrK8HHuPZ90-GRJMxo8CFg5qbGUCpMoDJKpeBaKkFkKjtaSHqC6rP5zJwibPO4EFGwQKY22BIysmsCg2cgMprJJmqvcUyUIxW32hbTpAgu_CgB5BOLfOKQb6KbqsVnSajxh23Dgl7ZObzPfv98jnagcVjW0l6ger5YmUu0rb7yyXJxVYyWb91HukQ |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LT8JAEJ4QNFEPvtCIzz14MbFQdrfb1hsRESMSDBi5Nd1HlQTBQPH3u1uWysGYeGmaZto0375mdma_D-CSM8wlpp4TyIA7VCXYiYVwncRlMiFM-okVm_A7nWAwCLsFuM7PwiilsuIzVTG3WS5fTsTcbJVVtavihUQH6GtGOcue1sp3VFxC9GwcZmlkahL6HlnmJd2w2u_dm0IuVsHahhpt4pV1aEVYJVtXmjv_-6Nd2Lb-I6ovGnwPCmq8D1srrIIleK6jZjxLUX9Jz4q0Y4p6H7qTIKOKjn5KXm5QvdFFr8P0XVtnHFUj1LCSKUIhS776dgAvzbv-bcuxygmOwIykTqxHHRY6dGMxE0aEjzLpuqrGQo6pktqL0GM3VooQX3kiIYT7THLhYx7zmvQ5OYTieDJWR4BMJlfHFNTjsQm3fB6aWYHqq-cnJOFlqC5xjISlFTfqFqMoCy_cMNLIRwb5yCJfhqv8jc8FpcYftiUDem5n8T7-_fEFbLT6T-2o_dB5PIFN_aFgUVl7CsV0OldnsC6-0uFsep71nG_DVr2N |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Fast+Technique+for+Smart+Home+Management%3A+ADP+With+Temporal+Difference+Learning&rft.jtitle=IEEE+transactions+on+smart+grid&rft.au=Keerthisinghe%2C+Chanaka&rft.au=Verbic%2C+Gregor&rft.au=Chapman%2C+Archie+C.&rft.date=2018-07-01&rft.issn=1949-3053&rft.eissn=1949-3061&rft.volume=9&rft.issue=4&rft.spage=3291&rft.epage=3303&rft_id=info:doi/10.1109%2FTSG.2016.2629470&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TSG_2016_2629470 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1949-3053&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1949-3053&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1949-3053&client=summon |