ED-DQN: An event-driven deep reinforcement learning control method for multi-zone residential buildings
Residential Heating, Ventilation, and Air conditioning (HVAC) systems are responsible for a significant amount of energy consumption, but their management is challenging due to the complexities of building thermodynamics and human activities. Reinforcement learning (RL) has been adopted to tackle th...
Saved in:
| Published in: | Building and environment Vol. 242; p. 110546 |
|---|---|
| Main Authors: | , , , , , , |
| Format: | Journal Article |
| Language: | English |
| Published: |
Elsevier Ltd
15.08.2023
|
| Subjects: | |
| ISSN: | 0360-1323 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | Residential Heating, Ventilation, and Air conditioning (HVAC) systems are responsible for a significant amount of energy consumption, but their management is challenging due to the complexities of building thermodynamics and human activities. Reinforcement learning (RL) has been adopted to tackle this issue, but traditional RL methods require massive training data, long learning periods, and frequent equipment adjustments. To address these issues, we construct a new event-driven Markov decision process (ED-MDP) framework, which enables adjustments of control policies triggered by events, reducing unnecessary operations. Moreover, we propose an event-driven deep Q network (ED-DQN) method, which optimizes the action selection based on the triggered events. In the HVAC control problem, the proposed ED-DQN can effectively capture dynamic non-linear features of thermal comfort, and reduce the equipment damage caused by frequent adjustments. Our experimental results show that compared to three benchmark methods and three RL methods, our ED-DQN achieved state-of-the-art performance in both energy saving and thermal comfort violations. Moreover, our method demonstrates promising performance when applied to new test thermal environments, indicating its robustness and adaptability for optimizing residential HVAC controls.
•An event-driven Markov decision processes framework for optimal HVAC control.•Two types of events capture factors impacting performance.•Event-driven Deep Q network improves learning speed and minimizes decisions.•Comprehensive experiments show the superiority of the proposed method. |
|---|---|
| AbstractList | Residential Heating, Ventilation, and Air conditioning (HVAC) systems are responsible for a significant amount of energy consumption, but their management is challenging due to the complexities of building thermodynamics and human activities. Reinforcement learning (RL) has been adopted to tackle this issue, but traditional RL methods require massive training data, long learning periods, and frequent equipment adjustments. To address these issues, we construct a new event-driven Markov decision process (ED-MDP) framework, which enables adjustments of control policies triggered by events, reducing unnecessary operations. Moreover, we propose an event-driven deep Q network (ED-DQN) method, which optimizes the action selection based on the triggered events. In the HVAC control problem, the proposed ED-DQN can effectively capture dynamic non-linear features of thermal comfort, and reduce the equipment damage caused by frequent adjustments. Our experimental results show that compared to three benchmark methods and three RL methods, our ED-DQN achieved state-of-the-art performance in both energy saving and thermal comfort violations. Moreover, our method demonstrates promising performance when applied to new test thermal environments, indicating its robustness and adaptability for optimizing residential HVAC controls.
•An event-driven Markov decision processes framework for optimal HVAC control.•Two types of events capture factors impacting performance.•Event-driven Deep Q network improves learning speed and minimizes decisions.•Comprehensive experiments show the superiority of the proposed method. |
| ArticleNumber | 110546 |
| Author | Luo, Jun Wang, Yunzhe Ding, Zhengkai Li, Zhu Fu, Qiming Chen, Jianping Lu, You |
| Author_xml | – sequence: 1 givenname: Qiming surname: Fu fullname: Fu, Qiming organization: School of Electronic and Information Engineering, Suzhou University of Science and Technology, Suzhou, Jiangsu, 215009, China – sequence: 2 givenname: Zhu surname: Li fullname: Li, Zhu organization: School of Electronic and Information Engineering, Suzhou University of Science and Technology, Suzhou, Jiangsu, 215009, China – sequence: 3 givenname: Zhengkai surname: Ding fullname: Ding, Zhengkai organization: School of Electronic and Information Engineering, Suzhou University of Science and Technology, Suzhou, Jiangsu, 215009, China – sequence: 4 givenname: Jianping orcidid: 0000-0002-2109-5761 surname: Chen fullname: Chen, Jianping email: alanjpchen@aliyun.com organization: School of Architecture and Environment, Sichuan University, Chengdu, Sichuan, 610065, China – sequence: 5 givenname: Jun surname: Luo fullname: Luo, Jun organization: School of Architecture and Environment, Sichuan University, Chengdu, Sichuan, 610065, China – sequence: 6 givenname: Yunzhe surname: Wang fullname: Wang, Yunzhe organization: School of Electronic and Information Engineering, Suzhou University of Science and Technology, Suzhou, Jiangsu, 215009, China – sequence: 7 givenname: You surname: Lu fullname: Lu, You organization: School of Electronic and Information Engineering, Suzhou University of Science and Technology, Suzhou, Jiangsu, 215009, China |
| BookMark | eNqFkM1OwzAQhH0oEm3hFZBfIME_qZsgDlRt-ZEqEBKcLcdeF1eJUzluJXh6XAoXLj2tVjvfaHZGaOA7DwhdUZJTQsX1Jq93rjHg9zkjjOeUkkkhBmhIuCAZ5Yyfo1Hfb0gSV7wYovVykS1en2_wzGPYg4-ZCS5NbAC2OIDztgsa2nTBDajgnV9j3fkYuga3ED86g5MCt7smuuwrpUlQ71KC6FSDf9IkpL9AZ1Y1PVz-zjF6v1--zR-z1cvD03y2yjSnLGbUCtDMKltaLQqjCltPyooZWxtiKmtIWq1ihoI2JTCgNQMxVZxMKzCiLPkY3R59dej6PoCV2kUV3SGxco2kRB6Kkhv5V5Q8FCWPRSVc_MO3wbUqfJ4G744gpOf2DoLstQOvwbgAOkrTuVMW31JAjqI |
| CitedBy_id | crossref_primary_10_3390_axioms12100908 crossref_primary_10_1016_j_enbuild_2024_114808 crossref_primary_10_1016_j_jobe_2024_110411 crossref_primary_10_1007_s10668_024_04968_w crossref_primary_10_1016_j_heliyon_2024_e25848 crossref_primary_10_1016_j_jclepro_2024_142507 crossref_primary_10_3390_s25092904 crossref_primary_10_1016_j_conbuildmat_2023_133522 crossref_primary_10_3390_en17010083 crossref_primary_10_1063_5_0276384 crossref_primary_10_1007_s00500_024_09637_8 crossref_primary_10_1016_j_energy_2024_132740 crossref_primary_10_1007_s00521_023_09236_y crossref_primary_10_1016_j_jobe_2023_108044 crossref_primary_10_1016_j_jobe_2024_109031 crossref_primary_10_1016_j_seta_2023_103502 crossref_primary_10_1016_j_energy_2023_128960 crossref_primary_10_1007_s10723_023_09701_x crossref_primary_10_1016_j_applthermaleng_2024_124987 crossref_primary_10_3390_en18061408 crossref_primary_10_1007_s10723_023_09704_8 crossref_primary_10_1016_j_applthermaleng_2025_126106 crossref_primary_10_1007_s10723_024_09749_3 crossref_primary_10_1016_j_resourpol_2023_104220 crossref_primary_10_1007_s11356_023_30118_2 crossref_primary_10_1016_j_enbuild_2024_115105 crossref_primary_10_1016_j_engappai_2023_107794 crossref_primary_10_3390_app131910717 crossref_primary_10_1016_j_enbuild_2025_116496 crossref_primary_10_1063_5_0179190 crossref_primary_10_3390_app14188479 crossref_primary_10_1016_j_rser_2023_114101 crossref_primary_10_1016_j_jobe_2025_113026 crossref_primary_10_1007_s11630_023_1933_5 crossref_primary_10_1016_j_phycom_2023_102188 crossref_primary_10_3390_pr12010156 crossref_primary_10_1016_j_enbuild_2025_115954 crossref_primary_10_1016_j_jobe_2024_110085 crossref_primary_10_1016_j_enbuild_2025_116045 crossref_primary_10_1016_j_comnet_2023_110105 crossref_primary_10_1016_j_eurpolymj_2024_112777 crossref_primary_10_1016_j_jobe_2023_108340 crossref_primary_10_1002_cpe_7995 crossref_primary_10_1002_sd_3073 crossref_primary_10_1016_j_applthermaleng_2024_123318 crossref_primary_10_1002_ett_70102 crossref_primary_10_3390_math11234744 crossref_primary_10_1016_j_enbuild_2025_115789 crossref_primary_10_1016_j_enbuild_2025_116037 crossref_primary_10_1016_j_buildenv_2024_111197 crossref_primary_10_1007_s12273_025_1231_0 crossref_primary_10_1016_j_enbuild_2024_115241 crossref_primary_10_1016_j_jobe_2024_111080 |
| Cites_doi | 10.1016/j.enbuild.2018.03.051 10.1016/j.apenergy.2018.09.188 10.1016/j.applthermaleng.2022.118552 10.1016/j.cie.2023.109112 10.1016/j.scitotenv.2020.138778 10.1016/j.enbuild.2022.112284 10.1109/JIOT.2020.2992117 10.3390/app112311162 10.1016/j.energy.2021.120741 10.1007/s12273-008-8118-8 10.1016/j.enbuild.2011.10.021 10.1016/j.apenergy.2020.116223 10.1016/j.buildenv.2022.109104 10.1016/j.enbuild.2017.07.008 10.1016/j.buildenv.2019.03.038 10.1016/j.engappai.2022.105485 10.1109/JIOT.2021.3078462 10.1109/ACCESS.2021.3054909 10.1038/nature14236 10.1016/j.apenergy.2020.116117 10.1016/j.enbuild.2019.02.029 10.1016/j.applthermaleng.2022.118442 10.1016/j.enbuild.2022.112269 10.1109/TASE.2018.2844258 |
| ContentType | Journal Article |
| Copyright | 2023 Elsevier Ltd |
| Copyright_xml | – notice: 2023 Elsevier Ltd |
| DBID | AAYXX CITATION |
| DOI | 10.1016/j.buildenv.2023.110546 |
| DatabaseName | CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| ExternalDocumentID | 10_1016_j_buildenv_2023_110546 S0360132323005735 |
| GroupedDBID | --K --M -~X .~1 0R~ 1B1 1RT 1~. 1~5 23N 4.4 457 4G. 5GY 5VS 7-5 71M 8P~ 9JM 9JN AABNK AACTN AAEDT AAEDW AAHCO AAIKC AAIKJ AAKOC AALRI AAMNW AAOAW AAQFI AARJD AAXKI AAXUO ABFNM ABFYP ABJNI ABLST ABMAC ACDAQ ACGFS ACIWK ACRLP ADBBV ADEZE ADTZH AEBSH AECPX AEKER AENEX AFJKZ AFKWA AFRAH AFTJW AFXIZ AGHFR AGUBO AGYEJ AHEUO AHHHB AHIDL AHJVU AIEXJ AIKHN AITUG AJOXV AKIFW AKRWK ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AXJTR BELTK BJAXD BKOJK BLECG BLXMC CS3 DU5 EBS EFJIC EO8 EO9 EP2 EP3 FDB FIRID FNPLU FYGXN G-Q GBLVA IHE J1W JARJE JJJVA KCYFY KOM LY6 LY7 LY9 M41 MO0 N9A O-L O9- OAUVE OZT P-8 P-9 PC. Q38 RNS ROL RPZ SDF SDG SDP SEN SES SPC SPCBC SSJ SSR SST SSZ T5K ~G- 9DU AAQXK AATTM AAYWO AAYXX ABWVN ABXDB ACLOT ACNNM ACRPL ACVFH ADCNI ADMUD ADNMO AEGFY AEIPS AEUPX AFPUW AGQPQ AI. AIGII AIIUN AKBMS AKYEP ANKPU APXCP ASPBG AVWKF AZFZN CITATION EFKBS EFLBG EJD FEDTE FGOYB G-2 HMC HVGLF HZ~ R2- SAC SET SEW VH1 WUQ ZMT ~HD |
| ID | FETCH-LOGICAL-c312t-1f6ec2faf8fc64da4fb5892dfbd0d9fd0b58fa2d1ecd8e2e1b2e67a3079ed6883 |
| ISICitedReferencesCount | 64 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001028981200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0360-1323 |
| IngestDate | Sat Nov 29 07:25:21 EST 2025 Tue Nov 18 21:19:42 EST 2025 Tue Dec 03 03:45:06 EST 2024 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Deep reinforcement learning Multi-zone residential buildings HVAC systems Event-driven Thermal comfort control |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c312t-1f6ec2faf8fc64da4fb5892dfbd0d9fd0b58fa2d1ecd8e2e1b2e67a3079ed6883 |
| ORCID | 0000-0002-2109-5761 |
| ParticipantIDs | crossref_citationtrail_10_1016_j_buildenv_2023_110546 crossref_primary_10_1016_j_buildenv_2023_110546 elsevier_sciencedirect_doi_10_1016_j_buildenv_2023_110546 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-08-15 |
| PublicationDateYYYYMMDD | 2023-08-15 |
| PublicationDate_xml | – month: 08 year: 2023 text: 2023-08-15 day: 15 |
| PublicationDecade | 2020 |
| PublicationTitle | Building and environment |
| PublicationYear | 2023 |
| Publisher | Elsevier Ltd |
| Publisher_xml | – name: Elsevier Ltd |
| References | Mozer, House (bib14) 1988 Fu, Han, Chen, Lu, Wu, Wang (bib7) 2022; 50 Ambroziak, Chojecki (bib5) 2023; 117 Yan, Xia, Tang, Song, Zhang, Jiang (bib33) 2008; 1 Wu, Jia, Guan (bib36) 2015 Qi, Xiao, Shi, Ward, Chen, Tu, Zhang (bib3) 2020; 728 Wang, Wang (bib10) 2021; 230 Bird, Daveau, O'Dwyer, Acha, Shah (bib11) 2022; 270 Chen, Norford, Samuelson, Malkawi (bib15) 2018; 169 Iqbal, Tham, Chang (bib29) 2021; 9 Yu, Qin, Zhang, Shen, Jiang, Guan (bib13) 2021; 8 Zeng, Barooah (bib12) 2021 Hamilton, Rapf, Kockat, Zuhaib, Abergel, Oppermann, Steurer (bib1) 2020 (bib31) 2005 Delač, Pavković, Lenić, Mađerić (bib4) 2022; 211 Jia, Wu, Wu, Guan (bib22) 2018; 15 Luo, Zhao, Zhu, Sun (bib28) 2023 Wang, Zhao (bib21) 2022; 218 Li, Gong, Fan, Peng, Chun, Fang (bib2) 2021; 282 Du, Zandi, Kotevska, Kurte, Munk, Amasyali, Li (bib19) 2021; 281 Sutton, Barto (bib24) 1998 Wemhoff (bib8) 2012; 45 Gao, Li, Wen (bib17) 2020; 7 Baldi, Korkas, Lv, Kosmatopoulos (bib9) 2018; 231 Fu, Chen, Ma, Ma, Fang, Xing, Chen (bib6) 2022; 270 Fang, Gong, Li, Chun, Peng, Li, Chen (bib18) 2022; 212 Wang, Hou, Chen, Fu, Huang (bib20) 2021; 39 Gu, Sung (bib27) 2021; 11 Wang, Huang, Sun, Liu (bib34) 2016; 133 Xu, Hu, Spanos, Schiavon (bib35) 2017; 152 Jin, Fu, Chen, Wang, Liu, Lu, Wu (bib26) 2023; 63 Valladares, Galindo, Gutiérrez, Wu, Liao, Liao, Wang (bib16) 2019; 155 Mnih, Kavukcuoglu, Silver, Rusu, Veness, Bellemare, Hassabis (bib25) 2015; 518 Jiang (bib32) 1982; 88 Deng, Yao, Yu, Zhang, Li (bib30) 2019; 190 Jiang (10.1016/j.buildenv.2023.110546_bib32) 1982; 88 Zeng (10.1016/j.buildenv.2023.110546_bib12) 2021 Fang (10.1016/j.buildenv.2023.110546_bib18) 2022; 212 Yan (10.1016/j.buildenv.2023.110546_bib33) 2008; 1 Delač (10.1016/j.buildenv.2023.110546_bib4) 2022; 211 Valladares (10.1016/j.buildenv.2023.110546_bib16) 2019; 155 Wang (10.1016/j.buildenv.2023.110546_bib34) 2016; 133 Du (10.1016/j.buildenv.2023.110546_bib19) 2021; 281 Wang (10.1016/j.buildenv.2023.110546_bib21) 2022; 218 Mnih (10.1016/j.buildenv.2023.110546_bib25) 2015; 518 Fu (10.1016/j.buildenv.2023.110546_bib7) 2022; 50 Hamilton (10.1016/j.buildenv.2023.110546_bib1) 2020 Baldi (10.1016/j.buildenv.2023.110546_bib9) 2018; 231 Mozer (10.1016/j.buildenv.2023.110546_bib14) 1988 Luo (10.1016/j.buildenv.2023.110546_bib28) 2023 Fu (10.1016/j.buildenv.2023.110546_bib6) 2022; 270 Wu (10.1016/j.buildenv.2023.110546_bib36) 2015 Deng (10.1016/j.buildenv.2023.110546_bib30) 2019; 190 Ambroziak (10.1016/j.buildenv.2023.110546_bib5) 2023; 117 Li (10.1016/j.buildenv.2023.110546_bib2) 2021; 282 Bird (10.1016/j.buildenv.2023.110546_bib11) 2022; 270 Jin (10.1016/j.buildenv.2023.110546_bib26) 2023; 63 Wemhoff (10.1016/j.buildenv.2023.110546_bib8) 2012; 45 Jia (10.1016/j.buildenv.2023.110546_bib22) 2018; 15 Gao (10.1016/j.buildenv.2023.110546_bib17) 2020; 7 Wang (10.1016/j.buildenv.2023.110546_bib20) 2021; 39 Gu (10.1016/j.buildenv.2023.110546_bib27) 2021; 11 Chen (10.1016/j.buildenv.2023.110546_bib15) 2018; 169 Yu (10.1016/j.buildenv.2023.110546_bib13) 2021; 8 (10.1016/j.buildenv.2023.110546_bib31) 2005 Qi (10.1016/j.buildenv.2023.110546_bib3) 2020; 728 Wang (10.1016/j.buildenv.2023.110546_bib10) 2021; 230 Xu (10.1016/j.buildenv.2023.110546_bib35) 2017; 152 Sutton (10.1016/j.buildenv.2023.110546_bib24) 1998 Iqbal (10.1016/j.buildenv.2023.110546_bib29) 2021; 9 |
| References_xml | – volume: 231 start-page: 1246 year: 2018 end-page: 1258 ident: bib9 article-title: Automating occupant-building interaction via smart zoning of thermostatic loads: a switched self-tuning approach[J] publication-title: Appl. Energy – volume: 211 year: 2022 ident: bib4 article-title: Integrated optimization of the building envelope and the HVAC system in nZEB refurbishment[J] publication-title: Appl. Therm. Eng. – year: 2021 ident: bib12 article-title: An Adaptive MPC Scheme for Energy-Efficient Control of Building HVAC Systems – volume: 270 year: 2022 ident: bib11 article-title: Real-world implementation and cost of a cloud-based MPC retrofit for HVAC control systems in commercial buildings[J] publication-title: Energy Build. – year: 2005 ident: bib31 article-title: China Standard Weather Data for Analyzing Building Thermal conditions[S] – year: 2020 ident: bib1 article-title: Global Status Report for Buildings and construction[J] – volume: 281 year: 2021 ident: bib19 article-title: Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning[J] publication-title: Appl. Energy – volume: 1 start-page: 95 year: 2008 end-page: 110 ident: bib33 article-title: DeST — an integrated building simulation toolkit Part I: fundamentals[J] publication-title: Build. Simulat. – volume: 9 start-page: 20440 year: 2021 end-page: 20449 ident: bib29 article-title: Double deep Q-network-based energy-efficient resource allocation in cloud radio access network[J] publication-title: IEEE Access – volume: 117 year: 2023 ident: bib5 article-title: The PID controller optimisation module using Fuzzy Self-Tuning PSO for Air Handling Unit in continuous operation[J] publication-title: Eng. Appl. Artif. Intell. – volume: 50 year: 2022 ident: bib7 article-title: Applications of reinforcement learning for building energy efficiency control: a review[J] publication-title: J. Build. Eng. – volume: 8 start-page: 12046 year: 2021 end-page: 12063 ident: bib13 article-title: A review of deep reinforcement learning for smart building energy management[J] publication-title: IEEE Internet Things J. – volume: 133 start-page: 79 year: 2016 end-page: 87 ident: bib34 article-title: Event-driven optimization of complex HVAC systems[J] publication-title: Energy Build. – volume: 155 start-page: 105 year: 2019 end-page: 117 ident: bib16 article-title: Energy optimization associated with thermal comfort and indoor air control via a deep reinforcement learning algorithm[J] publication-title: Building and Environment – volume: 45 start-page: 60 year: 2012 end-page: 66 ident: bib8 article-title: Calibration of HVAC equipment PID coefficients for energy conservation[J] publication-title: Energy Build. – volume: 152 start-page: 73 year: 2017 end-page: 85 ident: bib35 article-title: PMV-based event-triggered mechanism for building energy management under uncertainties[J] publication-title: Energy Build. – volume: 7 start-page: 8472 year: 2020 end-page: 8484 ident: bib17 article-title: Deepcomfort: energy-efficient thermal comfort control in buildings via reinforcement learning[J] publication-title: IEEE Internet Things J. – year: 1998 ident: bib24 article-title: Reinforcement Learning: an Introduction – volume: 169 start-page: 195 year: 2018 end-page: 205 ident: bib15 article-title: Optimal control of HVAC and window systems for natural ventilation through reinforcement learning[J] publication-title: Energy Build. – volume: 190 start-page: 155 year: 2019 end-page: 171 ident: bib30 article-title: Effectiveness of the thermal mass of external walls on residential buildings for part-time part-space heating and cooling using the state-space method[J] publication-title: Energy Build. – volume: 282 year: 2021 ident: bib2 article-title: A clustering-based approach for “cross-scale” load prediction on building level in HVAC systems[J] publication-title: Appl. Energy – volume: 88 start-page: 122 year: 1982 end-page: 141 ident: bib32 article-title: State-space method for the calculation of air-conditioning loads and the simulation of thermal behavior of the room[J] publication-title: Build. Eng. – volume: 518 start-page: 529 year: 2015 end-page: 533 ident: bib25 article-title: Human-level control through deep reinforcement learning[J] publication-title: Nature – volume: 218 year: 2022 ident: bib21 article-title: Event-driven online decoupling control mechanism for the variable flow rate HVAC system based on the medium response properties[J] publication-title: Build. Environ. – volume: 15 start-page: 1909 year: 2018 end-page: 1919 ident: bib22 article-title: Event-based HVAC control—a complexity-based approach[J] publication-title: IEEE Trans. Autom. Sci. Eng. – volume: 230 year: 2021 ident: bib10 article-title: A hierarchical optimal control strategy for continuous demand response of building HVAC systems to provide frequency regulation service to smart power grids publication-title: J. Energy – volume: 11 year: 2021 ident: bib27 article-title: Enhanced DQN framework for selecting actions and updating replay memory considering massive non-executable actions[J] publication-title: Appl. Sci. – volume: 39 year: 2021 ident: bib20 article-title: Data mining approach for improving the optimal control of HVAC systems: an event-driven strategy[J] publication-title: J. Build. Eng. – volume: 212 year: 2022 ident: bib18 article-title: Deep reinforcement learning optimal control strategy for temperature setpoint real-time reset in multi-zone building HVAC system[J] publication-title: Appl. Therm. Eng. – volume: 728 year: 2020 ident: bib3 article-title: COVID-19 transmission in Mainland China is associated with temperature and humidity: a time-series analysis[J] publication-title: Sci. Total Environ. – start-page: 10 year: 1988 end-page: 11 ident: bib14 article-title: An Environment that Adapts to its Inhabitants[C]//AAAI Spring Symposium on Intelligent Environments – volume: 270 year: 2022 ident: bib6 article-title: Optimal control method of HVAC based on multi-agent deep reinforcement learning[J] publication-title: Energy Build. – year: 2023 ident: bib28 article-title: A* guiding DQN algorithm for automated guided vehicle pathfinding problem of robotic mobile fulfillment systems[J] publication-title: Comput. Ind. Eng. – year: 2015 ident: bib36 article-title: Optimal Control of Multiroom HVAC System: an Event-Based Approach[J] – volume: 63 year: 2023 ident: bib26 article-title: A novel building energy consumption prediction method using deep reinforcement learning with consideration of fluctuation points[J] publication-title: J. Build. Eng. – volume: 63 year: 2023 ident: 10.1016/j.buildenv.2023.110546_bib26 article-title: A novel building energy consumption prediction method using deep reinforcement learning with consideration of fluctuation points[J] publication-title: J. Build. Eng. – volume: 169 start-page: 195 year: 2018 ident: 10.1016/j.buildenv.2023.110546_bib15 article-title: Optimal control of HVAC and window systems for natural ventilation through reinforcement learning[J] publication-title: Energy Build. doi: 10.1016/j.enbuild.2018.03.051 – volume: 231 start-page: 1246 year: 2018 ident: 10.1016/j.buildenv.2023.110546_bib9 article-title: Automating occupant-building interaction via smart zoning of thermostatic loads: a switched self-tuning approach[J] publication-title: Appl. Energy doi: 10.1016/j.apenergy.2018.09.188 – volume: 212 year: 2022 ident: 10.1016/j.buildenv.2023.110546_bib18 article-title: Deep reinforcement learning optimal control strategy for temperature setpoint real-time reset in multi-zone building HVAC system[J] publication-title: Appl. Therm. Eng. doi: 10.1016/j.applthermaleng.2022.118552 – year: 2023 ident: 10.1016/j.buildenv.2023.110546_bib28 article-title: A* guiding DQN algorithm for automated guided vehicle pathfinding problem of robotic mobile fulfillment systems[J] publication-title: Comput. Ind. Eng. doi: 10.1016/j.cie.2023.109112 – volume: 728 year: 2020 ident: 10.1016/j.buildenv.2023.110546_bib3 article-title: COVID-19 transmission in Mainland China is associated with temperature and humidity: a time-series analysis[J] publication-title: Sci. Total Environ. doi: 10.1016/j.scitotenv.2020.138778 – volume: 270 year: 2022 ident: 10.1016/j.buildenv.2023.110546_bib6 article-title: Optimal control method of HVAC based on multi-agent deep reinforcement learning[J] publication-title: Energy Build. doi: 10.1016/j.enbuild.2022.112284 – year: 2005 ident: 10.1016/j.buildenv.2023.110546_bib31 – volume: 7 start-page: 8472 issue: 9 year: 2020 ident: 10.1016/j.buildenv.2023.110546_bib17 article-title: Deepcomfort: energy-efficient thermal comfort control in buildings via reinforcement learning[J] publication-title: IEEE Internet Things J. doi: 10.1109/JIOT.2020.2992117 – volume: 88 start-page: 122 year: 1982 ident: 10.1016/j.buildenv.2023.110546_bib32 article-title: State-space method for the calculation of air-conditioning loads and the simulation of thermal behavior of the room[J] publication-title: Build. Eng. – volume: 11 issue: 23 year: 2021 ident: 10.1016/j.buildenv.2023.110546_bib27 article-title: Enhanced DQN framework for selecting actions and updating replay memory considering massive non-executable actions[J] publication-title: Appl. Sci. doi: 10.3390/app112311162 – volume: 230 year: 2021 ident: 10.1016/j.buildenv.2023.110546_bib10 article-title: A hierarchical optimal control strategy for continuous demand response of building HVAC systems to provide frequency regulation service to smart power grids publication-title: J. Energy doi: 10.1016/j.energy.2021.120741 – volume: 1 start-page: 95 issue: 2 year: 2008 ident: 10.1016/j.buildenv.2023.110546_bib33 article-title: DeST — an integrated building simulation toolkit Part I: fundamentals[J] publication-title: Build. Simulat. doi: 10.1007/s12273-008-8118-8 – volume: 39 year: 2021 ident: 10.1016/j.buildenv.2023.110546_bib20 article-title: Data mining approach for improving the optimal control of HVAC systems: an event-driven strategy[J] publication-title: J. Build. Eng. – volume: 45 start-page: 60 year: 2012 ident: 10.1016/j.buildenv.2023.110546_bib8 article-title: Calibration of HVAC equipment PID coefficients for energy conservation[J] publication-title: Energy Build. doi: 10.1016/j.enbuild.2011.10.021 – volume: 282 year: 2021 ident: 10.1016/j.buildenv.2023.110546_bib2 article-title: A clustering-based approach for “cross-scale” load prediction on building level in HVAC systems[J] publication-title: Appl. Energy doi: 10.1016/j.apenergy.2020.116223 – volume: 218 year: 2022 ident: 10.1016/j.buildenv.2023.110546_bib21 article-title: Event-driven online decoupling control mechanism for the variable flow rate HVAC system based on the medium response properties[J] publication-title: Build. Environ. doi: 10.1016/j.buildenv.2022.109104 – volume: 50 year: 2022 ident: 10.1016/j.buildenv.2023.110546_bib7 article-title: Applications of reinforcement learning for building energy efficiency control: a review[J] publication-title: J. Build. Eng. – volume: 152 start-page: 73 year: 2017 ident: 10.1016/j.buildenv.2023.110546_bib35 article-title: PMV-based event-triggered mechanism for building energy management under uncertainties[J] publication-title: Energy Build. doi: 10.1016/j.enbuild.2017.07.008 – year: 2021 ident: 10.1016/j.buildenv.2023.110546_bib12 – volume: 155 start-page: 105 year: 2019 ident: 10.1016/j.buildenv.2023.110546_bib16 article-title: Energy optimization associated with thermal comfort and indoor air control via a deep reinforcement learning algorithm[J] publication-title: Building and Environment doi: 10.1016/j.buildenv.2019.03.038 – volume: 117 year: 2023 ident: 10.1016/j.buildenv.2023.110546_bib5 article-title: The PID controller optimisation module using Fuzzy Self-Tuning PSO for Air Handling Unit in continuous operation[J] publication-title: Eng. Appl. Artif. Intell. doi: 10.1016/j.engappai.2022.105485 – volume: 8 start-page: 12046 issue: 15 year: 2021 ident: 10.1016/j.buildenv.2023.110546_bib13 article-title: A review of deep reinforcement learning for smart building energy management[J] publication-title: IEEE Internet Things J. doi: 10.1109/JIOT.2021.3078462 – year: 2020 ident: 10.1016/j.buildenv.2023.110546_bib1 – start-page: 10 year: 1988 ident: 10.1016/j.buildenv.2023.110546_bib14 – volume: 9 start-page: 20440 year: 2021 ident: 10.1016/j.buildenv.2023.110546_bib29 article-title: Double deep Q-network-based energy-efficient resource allocation in cloud radio access network[J] publication-title: IEEE Access doi: 10.1109/ACCESS.2021.3054909 – volume: 518 start-page: 529 issue: 7540 year: 2015 ident: 10.1016/j.buildenv.2023.110546_bib25 article-title: Human-level control through deep reinforcement learning[J] publication-title: Nature doi: 10.1038/nature14236 – year: 2015 ident: 10.1016/j.buildenv.2023.110546_bib36 – volume: 281 year: 2021 ident: 10.1016/j.buildenv.2023.110546_bib19 article-title: Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning[J] publication-title: Appl. Energy doi: 10.1016/j.apenergy.2020.116117 – year: 1998 ident: 10.1016/j.buildenv.2023.110546_bib24 – volume: 190 start-page: 155 year: 2019 ident: 10.1016/j.buildenv.2023.110546_bib30 article-title: Effectiveness of the thermal mass of external walls on residential buildings for part-time part-space heating and cooling using the state-space method[J] publication-title: Energy Build. doi: 10.1016/j.enbuild.2019.02.029 – volume: 211 year: 2022 ident: 10.1016/j.buildenv.2023.110546_bib4 article-title: Integrated optimization of the building envelope and the HVAC system in nZEB refurbishment[J] publication-title: Appl. Therm. Eng. doi: 10.1016/j.applthermaleng.2022.118442 – volume: 133 start-page: 79 issue: DEC year: 2016 ident: 10.1016/j.buildenv.2023.110546_bib34 article-title: Event-driven optimization of complex HVAC systems[J] publication-title: Energy Build. – volume: 270 year: 2022 ident: 10.1016/j.buildenv.2023.110546_bib11 article-title: Real-world implementation and cost of a cloud-based MPC retrofit for HVAC control systems in commercial buildings[J] publication-title: Energy Build. doi: 10.1016/j.enbuild.2022.112269 – volume: 15 start-page: 1909 issue: 4 year: 2018 ident: 10.1016/j.buildenv.2023.110546_bib22 article-title: Event-based HVAC control—a complexity-based approach[J] publication-title: IEEE Trans. Autom. Sci. Eng. doi: 10.1109/TASE.2018.2844258 |
| SSID | ssj0016934 |
| Score | 2.5998018 |
| Snippet | Residential Heating, Ventilation, and Air conditioning (HVAC) systems are responsible for a significant amount of energy consumption, but their management is... |
| SourceID | crossref elsevier |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 110546 |
| SubjectTerms | Deep reinforcement learning Event-driven HVAC systems Multi-zone residential buildings Thermal comfort control |
| Title | ED-DQN: An event-driven deep reinforcement learning control method for multi-zone residential buildings |
| URI | https://dx.doi.org/10.1016/j.buildenv.2023.110546 |
| Volume | 242 |
| WOSCitedRecordID | wos001028981200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 issn: 0360-1323 databaseCode: AIEXJ dateStart: 19950101 customDbUrl: isFulltext: true dateEnd: 99991231 titleUrlDefault: https://www.sciencedirect.com omitProxy: false ssIdentifier: ssj0016934 providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1LbxMxELZC2gMcEJSilpd84Fa53dj7sLlFNAgQiqhapKiX1a4f6UvbKE2qil_P-LXZQqVSIS6rXSu2s57P3vF45huE3itTGcWKhFSpqQloxDDnpJSkSnidSUYT7SnzvxXjMZ9MxPdez8RYmOuLomn4zY2Y_VdRQxkI24bOPkDcbaNQAPcgdLiC2OH6V4If7ZP9g3Ew-Dl-JqLmdk3bUVrPdubacaVKZxaMSSOmrcu6zyjtnA-dryH5ednYzCo2q2ezsOb1OiTSvrp1HhwK3VlEJ3iuhcfSyvLAphCbtj5AzpHg-GTZKtMhv8rxiW6m59XpyvUgRJAAlGexgWCpoMyaXn2spjefxRCalb-SD9tKCGyJWXdJpp5x64_l3Vsaznbdm8LL7NpubCRDlv7Gp-2-0Ie2cds2ZY74MXuE1miRCd5Ha8Mvo8nX9rwpFywQjfk_04klv7u3u9WYjmpy9Aw9DXsKPPRYeI56utlATzpMky_Q1KPiAx42uIsJbDGBb2ECR0zggAnsMYHhF3iFCdzBBG4xsYl-fBodffxMQo4NItmALsjA5FpSmLDcyDxVdsZmXFBlapUoYVQCj6aiaqCl4prqQU11XlTwZRBa5Zyzl6jfQKdbCEtWJ4LnUtRFmhqe1dyy-dEiSWGTIKtsG2VxyEoZCOhtHpSLMnoanpVxqEs71KUf6m2019abeQqWe2uIKJEyKJJeQSwBSPfUffUPdV-jxyvcv0H9xXyp36J1eb04vZq_C5j7BWpbn80 |
| linkProvider | Elsevier |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=ED-DQN%3A+An+event-driven+deep+reinforcement+learning+control+method+for+multi-zone+residential+buildings&rft.jtitle=Building+and+environment&rft.au=Fu%2C+Qiming&rft.au=Li%2C+Zhu&rft.au=Ding%2C+Zhengkai&rft.au=Chen%2C+Jianping&rft.date=2023-08-15&rft.pub=Elsevier+Ltd&rft.issn=0360-1323&rft.volume=242&rft_id=info:doi/10.1016%2Fj.buildenv.2023.110546&rft.externalDocID=S0360132323005735 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0360-1323&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0360-1323&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0360-1323&client=summon |