Bayesian optimization for hyper-parameter tuning of an improved twin delayed deep deterministic policy gradients based energy management strategy for plug-in hybrid electric vehicles
Hybridization and electrification of vehicles are underway to achieve Net-zero emissions for road transport. The upcoming deep reinforcement learning (DRL) algorithm shows great promise for the efficient energy management of PHEVs, as it provides the potential to achieve theoretical optimal performa...
Gespeichert in:
| Veröffentlicht in: | Applied energy Jg. 381; S. 125171 |
|---|---|
| Hauptverfasser: | , , , , , , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
Elsevier Ltd
01.03.2025
|
| Schlagworte: | |
| ISSN: | 0306-2619 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | Hybridization and electrification of vehicles are underway to achieve Net-zero emissions for road transport. The upcoming deep reinforcement learning (DRL) algorithm shows great promise for the efficient energy management of PHEVs, as it provides the potential to achieve theoretical optimal performance. However, the brittle convergence properties, high sample complexity, and sensitivity to hyper-parameters of DRL algorithms have been major challenges in this field, limiting the applicability of DRL to real-world tasks. A novel EMS for PHEV based on Bayesian Optimization (BO) and improved Twin Delay Deep Deterministic Policy Gradient (TD3) algorithm is proposed in this paper, in which BO is introduced to optimize the TD3 hyper-parameters and a non-parametric reward function (NRF) is designed to improve the TD3 algorithm (BO-NRTD3). The present work addresses two challenges to contribute to the proposed EMS: (1) By hyper-parameter tuning, the TD3 strategy’s brittle convergence and robustness characteristics have been significantly improved; and (2) By designing the non-parametric reward function (NRF), the TD3 strategy can tackle system uncertainties. These findings are validated by comparing with various cutting-edge DRL and DP strategies using Software-in-the-Loop (SiL) and Hardware-in-the-Loop (HiL) tests. The results show that the energy economy of the BO-NRTD3 strategy is up to 98.15% of DP and 4.23% more robust than the parametric reward function TD3 (PR-TD3) strategy.
•Improved TD3 strategy’s convergence and robustness via Bayesian Optimization.•Objective function ensures fast convergence of the reward function and its magnitude.•Non-parametric reward function enhances TD3’s adaptability to system uncertainties.•BO-NRTD3 strategy validated through Software-in-the-Loop and Hardware-in-the-Loop. |
|---|---|
| AbstractList | Hybridization and electrification of vehicles are underway to achieve Net-zero emissions for road transport. The upcoming deep reinforcement learning (DRL) algorithm shows great promise for the efficient energy management of PHEVs, as it provides the potential to achieve theoretical optimal performance. However, the brittle convergence properties, high sample complexity, and sensitivity to hyper-parameters of DRL algorithms have been major challenges in this field, limiting the applicability of DRL to real-world tasks. A novel EMS for PHEV based on Bayesian Optimization (BO) and improved Twin Delay Deep Deterministic Policy Gradient (TD3) algorithm is proposed in this paper, in which BO is introduced to optimize the TD3 hyper-parameters and a non-parametric reward function (NRF) is designed to improve the TD3 algorithm (BO-NRTD3). The present work addresses two challenges to contribute to the proposed EMS: (1) By hyper-parameter tuning, the TD3 strategy’s brittle convergence and robustness characteristics have been significantly improved; and (2) By designing the non-parametric reward function (NRF), the TD3 strategy can tackle system uncertainties. These findings are validated by comparing with various cutting-edge DRL and DP strategies using Software-in-the-Loop (SiL) and Hardware-in-the-Loop (HiL) tests. The results show that the energy economy of the BO-NRTD3 strategy is up to 98.15% of DP and 4.23% more robust than the parametric reward function TD3 (PR-TD3) strategy. Hybridization and electrification of vehicles are underway to achieve Net-zero emissions for road transport. The upcoming deep reinforcement learning (DRL) algorithm shows great promise for the efficient energy management of PHEVs, as it provides the potential to achieve theoretical optimal performance. However, the brittle convergence properties, high sample complexity, and sensitivity to hyper-parameters of DRL algorithms have been major challenges in this field, limiting the applicability of DRL to real-world tasks. A novel EMS for PHEV based on Bayesian Optimization (BO) and improved Twin Delay Deep Deterministic Policy Gradient (TD3) algorithm is proposed in this paper, in which BO is introduced to optimize the TD3 hyper-parameters and a non-parametric reward function (NRF) is designed to improve the TD3 algorithm (BO-NRTD3). The present work addresses two challenges to contribute to the proposed EMS: (1) By hyper-parameter tuning, the TD3 strategy’s brittle convergence and robustness characteristics have been significantly improved; and (2) By designing the non-parametric reward function (NRF), the TD3 strategy can tackle system uncertainties. These findings are validated by comparing with various cutting-edge DRL and DP strategies using Software-in-the-Loop (SiL) and Hardware-in-the-Loop (HiL) tests. The results show that the energy economy of the BO-NRTD3 strategy is up to 98.15% of DP and 4.23% more robust than the parametric reward function TD3 (PR-TD3) strategy. •Improved TD3 strategy’s convergence and robustness via Bayesian Optimization.•Objective function ensures fast convergence of the reward function and its magnitude.•Non-parametric reward function enhances TD3’s adaptability to system uncertainties.•BO-NRTD3 strategy validated through Software-in-the-Loop and Hardware-in-the-Loop. |
| ArticleNumber | 125171 |
| Author | Zhou, Quan Du, Changqing Wang, Jinhai Yuan, Quan Yan, Fuwu Hua, Min Gongye, Xiangyu Xu, Hongming |
| Author_xml | – sequence: 1 givenname: Jinhai orcidid: 0000-0002-0117-9597 surname: Wang fullname: Wang, Jinhai organization: School of Automotive Engineering, Wuhan University of Technology, Wuhan, Hubei, 430070, China – sequence: 2 givenname: Changqing orcidid: 0000-0003-1292-8773 surname: Du fullname: Du, Changqing organization: School of Automotive Engineering, Wuhan University of Technology, Wuhan, Hubei, 430070, China – sequence: 3 givenname: Fuwu orcidid: 0000-0002-4402-8676 surname: Yan fullname: Yan, Fuwu email: yanfuwu@vip.sina.com organization: School of Automotive Engineering, Wuhan University of Technology, Wuhan, Hubei, 430070, China – sequence: 4 givenname: Min surname: Hua fullname: Hua, Min organization: School of Engineering, University of Birmingham, Birmingham, B15 2TT, UK – sequence: 5 givenname: Xiangyu surname: Gongye fullname: Gongye, Xiangyu organization: School of Automotive Engineering, Wuhan University of Technology, Wuhan, Hubei, 430070, China – sequence: 6 givenname: Quan surname: Yuan fullname: Yuan, Quan organization: School of Automotive Engineering, Wuhan University of Technology, Wuhan, Hubei, 430070, China – sequence: 7 givenname: Hongming surname: Xu fullname: Xu, Hongming organization: School of Automotive Engineering, Wuhan University of Technology, Wuhan, Hubei, 430070, China – sequence: 8 givenname: Quan orcidid: 0000-0003-4216-3468 surname: Zhou fullname: Zhou, Quan organization: School of Engineering, University of Birmingham, Birmingham, B15 2TT, UK |
| BookMark | eNqFkTuP1DAUhV0sEvvgLyCXNBn8SJyJRAGseEkr0UBtOfZN5o4SO9ieQdkfxu_DQ6Ch2cZXsr9zbJ9zQ6588EDIS852nHH1-rgzC3iI47oTTNQ7Lhre8ityzSRTlVC8e05uUjoyxgQX7Jr8em9WSGg8DUvGGR9NxuDpECI9rAvEajHRzJAh0nzy6EcaBlponJcYzuBo_omeOpiKjSsTlrIUekaPKaOlS5jQrnSMxiH4nGhvUiG3R9LZeDPCXA5oytFkKHuXu5fpNFbF-LD2EQs9gc2xuJ3hgHaCdEeeDWZK8OLvvCXfP374dv-5evj66cv9u4fKyrrJlZJNK8QehOq5aZyorRIwtM5x29V1bYG1trGuUYN0tWxMD2ZQru86CS3fy17eklebb_ntjxOkrGdMFqbJeAinpCVXzV41ndwXVG2ojSGlCINeIs4mrpozfSlHH_W_cvSlHL2VU4Rv_hNazH9qKIng9LT87SaHksMZIepkS9IWHMaSmnYBn7L4DavDumU |
| CitedBy_id | crossref_primary_10_1016_j_energy_2025_137836 crossref_primary_10_1016_j_net_2025_103741 crossref_primary_10_3390_electronics14112251 crossref_primary_10_1016_j_apenergy_2025_125898 crossref_primary_10_1016_j_ijhydene_2025_151462 |
| Cites_doi | 10.1016/j.energy.2019.07.126 10.1016/j.enconman.2024.118853 10.1016/j.energy.2023.126858 10.1016/j.egyr.2022.04.050 10.1016/j.neucom.2020.07.061 10.3390/en16010074 10.1145/3451254 10.1016/j.energy.2022.126497 10.1109/TTE.2021.3056432 10.1016/j.energy.2022.125084 10.1016/j.enconman.2023.117480 10.1016/j.apenergy.2018.12.032 10.1016/j.energy.2021.120118 10.1038/s41598-024-54515-w 10.1016/j.jpowsour.2013.09.085 10.1016/j.ese.2021.100125 10.1504/IJVAS.2020.104818 10.1016/j.engappai.2023.106239 10.1016/j.enconman.2022.115450 10.1016/j.apenergy.2023.121227 10.1002/ente.202200355 10.1016/j.energy.2023.127207 10.1038/s43246-020-00095-x 10.1109/TTE.2020.3014688 10.1109/TTE.2022.3141191 10.1016/j.energy.2022.124806 10.1016/j.apenergy.2023.121526 10.1016/j.ijhydene.2021.12.121 10.3390/smartcities4010022 10.1049/iet-its.2019.0606 10.1109/TVT.2021.3107734 |
| ContentType | Journal Article |
| Copyright | 2024 |
| Copyright_xml | – notice: 2024 |
| DBID | AAYXX CITATION 7S9 L.6 |
| DOI | 10.1016/j.apenergy.2024.125171 |
| DatabaseName | CrossRef AGRICOLA AGRICOLA - Academic |
| DatabaseTitle | CrossRef AGRICOLA AGRICOLA - Academic |
| DatabaseTitleList | AGRICOLA |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Environmental Sciences |
| ExternalDocumentID | 10_1016_j_apenergy_2024_125171 S0306261924025558 |
| GroupedDBID | --K --M .~1 0R~ 1B1 1~. 1~5 23M 4.4 457 4G. 5GY 5VS 7-5 71M 8P~ 9JN AABNK AACTN AAEDT AAEDW AAHBH AAHCO AAIKJ AAKOC AALRI AAOAW AAQFI AARJD AATTM AAXKI AAXUO ABJNI ABMAC ACDAQ ACGFS ACRLP ADBBV ADEZE ADTZH AEBSH AECPX AEIPS AEKER AENEX AFJKZ AFTJW AGHFR AGUBO AGYEJ AHHHB AHIDL AHJVU AIEXJ AIKHN AITUG AKRWK ALMA_UNASSIGNED_HOLDINGS AMRAJ ANKPU AXJTR BELTK BJAXD BKOJK BLXMC BNPGV CS3 EBS EFJIC EO8 EO9 EP2 EP3 FDB FIRID FNPLU FYGXN G-Q GBLVA IHE J1W JARJE JJJVA KOM LY6 M41 MO0 N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. Q38 ROL RPZ SDF SDG SES SEW SPC SPCBC SSH SSR SST SSZ T5K TN5 ~02 ~G- 9DU AAQXK AAYWO AAYXX ABEFU ABFNM ABWVN ABXDB ACLOT ACNNM ACRPL ACVFH ADCNI ADMUD ADNMO AEUPX AFPUW AGQPQ AIGII AIIUN AKBMS AKYEP APXCP ASPBG AVWKF AZFZN CITATION EFKBS EFLBG EJD FEDTE FGOYB G-2 HVGLF HZ~ R2- SAC WUQ ZY4 ~HD 7S9 L.6 |
| ID | FETCH-LOGICAL-c345t-6357228e26b1a5d24c62ef7dd1c9444ce07c5cd56f3d435abeaf6db993e7183b3 |
| ISICitedReferencesCount | 7 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001398530400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0306-2619 |
| IngestDate | Fri Oct 03 00:15:09 EDT 2025 Sat Nov 29 08:18:43 EST 2025 Tue Nov 18 21:33:18 EST 2025 Sun Apr 06 06:53:53 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Twin Delayed Deep Deterministic Policy Gradients Energy management strategy Non-parametric reward function Bayesian optimization Plug-in hybrid electric vehicles |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c345t-6357228e26b1a5d24c62ef7dd1c9444ce07c5cd56f3d435abeaf6db993e7183b3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
| ORCID | 0000-0002-4402-8676 0000-0003-1292-8773 0000-0002-0117-9597 0000-0003-4216-3468 |
| PQID | 3165865938 |
| PQPubID | 24069 |
| ParticipantIDs | proquest_miscellaneous_3165865938 crossref_primary_10_1016_j_apenergy_2024_125171 crossref_citationtrail_10_1016_j_apenergy_2024_125171 elsevier_sciencedirect_doi_10_1016_j_apenergy_2024_125171 |
| PublicationCentury | 2000 |
| PublicationDate | 2025-03-01 2025-03-00 20250301 |
| PublicationDateYYYYMMDD | 2025-03-01 |
| PublicationDate_xml | – month: 03 year: 2025 text: 2025-03-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationTitle | Applied energy |
| PublicationYear | 2025 |
| Publisher | Elsevier Ltd |
| Publisher_xml | – name: Elsevier Ltd |
| References | Chen, Mi, Xiong, Xu, You (b15) 2014; 248 Hu, Han, Tang, Lin (b6) 2021; 7 Wang, Jin, Schmitt, Olhofer (b43) 2022 Bibra, Connelly, Dhir, Drtil, Henriot, Hwang (b5) 2022 Hua, Chen, Zong, He (b7) 2020; 15 Sun, Zou, Zhang, Guo, Zhang, Du (b44) 2022 Sundstrom, Guzzella (b13) 2009 Paganelli, Delprat, Guerra, Rimaux, Santin (b18) 2002; Vol. 4 Fujimoto, Hoof, Meger (b29) 2018 Yuan, Zou, Jung, Kim (b11) 2022; 47 Mei, Karimi, Xie, Chen, Huang, Yang (b27) 2023; 123 Chen, Hu, Guo (b14) 2023; 273 Gulde, Tuscher, Csiszar, Riedel, Verl (b36) 2020 Zhou, Xue, Xue, Liao, Liu, Zhao (b45) 2021; 224 Yang, Yin (b40) 2021; 4 Yang, Shami (b38) 2020; 415 Cawley, Talbot (b48) 2007; 8 Duan, Hu, Deng, Fang, Xiong, Lu (b4) 2021; 8 Zhang, Zhang, Hong, Zhang, Yang, Jia (b25) 2023; 269 Rioual, Moullec, Laurent, Khan, Diguet (b42) 2021 Lillicrap, Hunt, Pritzel, Heess, Erez, Tassa (b26) 2015 Yang, Liu, Wen (b41) 2024; 14 Li, Gao, Wu, Xia, Wang, Hu (b47) 2023; 293 Mabrey (b3) 2019 Hu, Xie, Song, Zhang, Yan (b24) 2023; 342 Xie, Hu, Xin, Brighton (b17) 2019; 236 Trinh, Truong, Do, Nguyen, Ahn (b21) 2022; 8 Makarova A, Shen H, Perrone V, Klein A, Faddoul JB, Krause A, et al. Overfitting in Bayesian optimization: an empirical study and early-stopping solution. In: 2nd workshop on neural architecture search (NAS 2021 collocated with the 9th ICLR 2021). 2021. Wang, Zeng, Song, Yang (b12) 2019; 185 Ehsani, Gao, Longo, Ebrahimi (b8) 2018 Hua, Shuai, Zhou, Wang, He, Xu (b23) 2023 Liessner, Schmitt, Dietermann, Bäker (b37) 2019 Zhang, Huang, Chen, Li, Liu (b19) 2022; 8 Wang, Ye, Zhang, Xu (b22) 2023; 266 Yang, Zha, Wang, Liu, Xiang (b32) 2020; 14 Yan, Wang, Du, Hua (b10) 2022; 16 Tang, Chen, Liu, Qin, Cao (b46) 2021; 70 Wu, Huang, Li, Li, Peng, Guerrero (b20) 2024; 317 Khalatbarisoltani, Zhou, Tang, Kandidayeni, Boulon, Hu (b9) 2023 Ramasamy, Chandramohan, Ghanta (b34) 2022; 10 Xu, Dai, Gaines, Hu, Tukker, Steubing (b1) 2020; 1 Wang, Du, Yan, Zhou, Xu (b28) 2023 Liessner, Lorenz, Schmitt, Dietermann, Baker (b39) 2019 Hua, Zhang, Zhang, Li, Yu, Xu (b30) 2023; 348 Wu, Ruan, Cui, Zhang, Li, Zhang (b31) 2023; 262 Sanguesa, Torres-Sanz, Garrido, Martinez, Marquez-Barja (b2) 2021; 4 Zhou, He, Zhao, Li, Li, Williams (b16) 2020; 7 Zhang, Liu, Lei, Fan, Li, Wang (b35) 2022; 257 Deng, Hai, Peng, Löwenstein, Hameyer (b33) 2021 Liessner (10.1016/j.apenergy.2024.125171_b37) 2019 Yan (10.1016/j.apenergy.2024.125171_b10) 2022; 16 Wang (10.1016/j.apenergy.2024.125171_b22) 2023; 266 Xu (10.1016/j.apenergy.2024.125171_b1) 2020; 1 Deng (10.1016/j.apenergy.2024.125171_b33) 2021 Hu (10.1016/j.apenergy.2024.125171_b24) 2023; 342 Yuan (10.1016/j.apenergy.2024.125171_b11) 2022; 47 Zhou (10.1016/j.apenergy.2024.125171_b45) 2021; 224 Sanguesa (10.1016/j.apenergy.2024.125171_b2) 2021; 4 Wu (10.1016/j.apenergy.2024.125171_b20) 2024; 317 Wang (10.1016/j.apenergy.2024.125171_b12) 2019; 185 Yang (10.1016/j.apenergy.2024.125171_b32) 2020; 14 Zhang (10.1016/j.apenergy.2024.125171_b35) 2022; 257 Wang (10.1016/j.apenergy.2024.125171_b43) 2022 Tang (10.1016/j.apenergy.2024.125171_b46) 2021; 70 Wu (10.1016/j.apenergy.2024.125171_b31) 2023; 262 Rioual (10.1016/j.apenergy.2024.125171_b42) 2021 Cawley (10.1016/j.apenergy.2024.125171_b48) 2007; 8 Xie (10.1016/j.apenergy.2024.125171_b17) 2019; 236 Sundstrom (10.1016/j.apenergy.2024.125171_b13) 2009 Zhang (10.1016/j.apenergy.2024.125171_b25) 2023; 269 Chen (10.1016/j.apenergy.2024.125171_b15) 2014; 248 Zhou (10.1016/j.apenergy.2024.125171_b16) 2020; 7 Li (10.1016/j.apenergy.2024.125171_b47) 2023; 293 Hua (10.1016/j.apenergy.2024.125171_b30) 2023; 348 Yang (10.1016/j.apenergy.2024.125171_b40) 2021; 4 Bibra (10.1016/j.apenergy.2024.125171_b5) 2022 Trinh (10.1016/j.apenergy.2024.125171_b21) 2022; 8 Liessner (10.1016/j.apenergy.2024.125171_b39) 2019 Zhang (10.1016/j.apenergy.2024.125171_b19) 2022; 8 Hua (10.1016/j.apenergy.2024.125171_b23) 2023 Yang (10.1016/j.apenergy.2024.125171_b38) 2020; 415 10.1016/j.apenergy.2024.125171_b49 Ehsani (10.1016/j.apenergy.2024.125171_b8) 2018 Khalatbarisoltani (10.1016/j.apenergy.2024.125171_b9) 2023 Lillicrap (10.1016/j.apenergy.2024.125171_b26) 2015 Chen (10.1016/j.apenergy.2024.125171_b14) 2023; 273 Paganelli (10.1016/j.apenergy.2024.125171_b18) 2002; Vol. 4 Mei (10.1016/j.apenergy.2024.125171_b27) 2023; 123 Fujimoto (10.1016/j.apenergy.2024.125171_b29) 2018 Hu (10.1016/j.apenergy.2024.125171_b6) 2021; 7 Sun (10.1016/j.apenergy.2024.125171_b44) 2022 Wang (10.1016/j.apenergy.2024.125171_b28) 2023 Mabrey (10.1016/j.apenergy.2024.125171_b3) 2019 Yang (10.1016/j.apenergy.2024.125171_b41) 2024; 14 Hua (10.1016/j.apenergy.2024.125171_b7) 2020; 15 Ramasamy (10.1016/j.apenergy.2024.125171_b34) 2022; 10 Gulde (10.1016/j.apenergy.2024.125171_b36) 2020 Duan (10.1016/j.apenergy.2024.125171_b4) 2021; 8 |
| References_xml | – year: 2023 ident: b23 article-title: Recent progress in energy management of connected hybrid electric vehicles using reinforcement learning – volume: 293 year: 2023 ident: b47 article-title: Incentive learning-based energy management for hybrid energy storage system in electric vehicles publication-title: Energy Convers Manage – year: 2022 ident: b5 article-title: Global EV outlook 2022: Securing supplies for an electric future – start-page: 1625 year: 2009 end-page: 1630 ident: b13 article-title: A generic dynamic programming matlab function publication-title: 2009 IEEE control applications,(CCA) & intelligent control – start-page: 1587 year: 2018 end-page: 1596 ident: b29 article-title: Addressing function approximation error in actor-critic methods publication-title: International conference on machine learning – year: 2019 ident: b3 article-title: Reducing the carbon footprint? How hybrid cars help – year: 2021 ident: b42 article-title: Design and comparison of reward functions in reinforcement learning for energy management of sensor nodes – volume: 8 year: 2021 ident: b4 article-title: Impacts of reducing air pollutants and CO2 emissions in urban road transport through 2035 in Chongqing, China publication-title: Environ Sci Ecotechnol – year: 2023 ident: b9 article-title: Energy management strategies for fuel cell vehicles: a comprehensive review of the latest progress in modeling, strategies, and future prospects publication-title: IEEE Trans Intell Transp Syst – volume: Vol. 4 start-page: 2076 year: 2002 end-page: 2081 ident: b18 article-title: Equivalent consumption minimization strategy for parallel hybrid powertrains publication-title: Vehicular technology conference. IEEE 55th vehicular technology conference. VTC spring 2002 (Cat. No. 02CH37367) – volume: 7 start-page: 1990 year: 2021 end-page: 2009 ident: b6 article-title: Powertrain design and control in electrified vehicles: A critical review publication-title: IEEE Trans Transp Electrif – volume: 185 start-page: 1086 year: 2019 end-page: 1099 ident: b12 article-title: Optimal rule design methodology for energy management strategy of a power-split hybrid electric bus publication-title: Energy – start-page: 1 year: 2021 end-page: 6 ident: b33 article-title: Deep reinforcement learning based energy management strategy for fuel cell and battery powered rail vehicles publication-title: 2021 IEEE vehicle power and propulsion conference – volume: 8 start-page: 2516 year: 2022 end-page: 2530 ident: b19 article-title: An optimal control strategy for plug-in hybrid electric vehicles based on enhanced model predictive control with efficient numerical method publication-title: IEEE Trans Transp Electrif – volume: 70 start-page: 9922 year: 2021 end-page: 9934 ident: b46 article-title: Distributed deep reinforcement learning-based energy and emission management strategy for hybrid electric vehicles publication-title: IEEE Trans Veh Technol – volume: 14 start-page: 3948 year: 2024 ident: b41 article-title: The impact of Bayesian optimization on feature selection publication-title: Sci Rep – year: 2018 ident: b8 article-title: Modern electric, hybrid electric, and fuel cell vehicles – year: 2022 ident: b43 article-title: Recent advances in Bayesian optimization – volume: 7 start-page: 616 year: 2020 end-page: 625 ident: b16 article-title: Modified particle swarm optimization with chaotic attraction strategy for modular design of hybrid powertrains publication-title: IEEE Trans Transp Electrif – year: 2015 ident: b26 article-title: Continuous control with deep reinforcement learning – volume: 15 start-page: 50 year: 2020 end-page: 76 ident: b7 article-title: Research on synchronous control strategy of steer-by-wire system with dual steering actuator motors publication-title: Int J Veh Auton Syst – start-page: 134 year: 2019 end-page: 144 ident: b37 article-title: Hyperparameter optimization for deep reinforcement learning in vehicle energy management publication-title: ICAART (2) – start-page: 32 year: 2020 end-page: 35 ident: b36 article-title: Deep reinforcement learning using cyclical learning rates publication-title: 2020 third international conference on artificial intelligence for industries (AI4I) – volume: 248 start-page: 416 year: 2014 end-page: 426 ident: b15 article-title: Energy management of a power-split plug-in hybrid electric vehicle based on genetic algorithm and quadratic programming publication-title: J Power Sources – volume: 8 start-page: 6035 year: 2022 end-page: 6057 ident: b21 article-title: Optimization-based energy management strategies for hybrid construction machinery: A review publication-title: Energy Rep – volume: 123 year: 2023 ident: b27 article-title: A deep reinforcement learning approach to energy management control with connected information for hybrid electric vehicles publication-title: Eng Appl Artif Intell – start-page: 1 year: 2019 end-page: 6 ident: b39 article-title: Simultaneous electric powertrain hardware and energy management optimization of a hybrid electric vehicle using deep reinforcement learning and Bayesian optimization publication-title: 2019 IEEE vehicle power and propulsion conference – volume: 415 start-page: 295 year: 2020 end-page: 316 ident: b38 article-title: On hyperparameter optimization of machine learning algorithms: Theory and practice publication-title: Neurocomputing – volume: 1 start-page: 99 year: 2020 ident: b1 article-title: Future material demand for automotive lithium-based batteries publication-title: Commun Mater – year: 2022 ident: b44 article-title: High robustness energy management strategy of hybrid electric vehicle based on improved soft actor-critic deep reinforcement learning publication-title: Energy – volume: 236 start-page: 893 year: 2019 end-page: 905 ident: b17 article-title: Pontryagin’s minimum principle based model predictive control of energy management for a plug-in hybrid electric bus publication-title: Appl Energy – reference: Makarova A, Shen H, Perrone V, Klein A, Faddoul JB, Krause A, et al. Overfitting in Bayesian optimization: an empirical study and early-stopping solution. In: 2nd workshop on neural architecture search (NAS 2021 collocated with the 9th ICLR 2021). 2021. – volume: 16 start-page: 74 year: 2022 ident: b10 article-title: Multi-objective energy management strategy for hybrid electric vehicles based on TD3 with non-parametric reward function publication-title: Energies – volume: 348 year: 2023 ident: b30 article-title: Energy management of multi-mode plug-in hybrid electric vehicle using multi-agent deep reinforcement learning publication-title: Appl Energy – volume: 4 start-page: 1 year: 2021 end-page: 19 ident: b40 article-title: Efficient hyperparameter optimization for physics-based character animation publication-title: Proc ACM Comput Graph Interact Tech – volume: 10 year: 2022 ident: b34 article-title: Energy management in plugin hybrid electric vehicles with hybrid energy storage system using hybrid approach publication-title: Energy Technol – volume: 273 year: 2023 ident: b14 article-title: Fast dynamic-programming algorithm for solving global optimization problems of hybrid electric vehicles publication-title: Energy – volume: 224 year: 2021 ident: b45 article-title: A novel energy management strategy of hybrid electric vehicle via an improved TD3 deep reinforcement learning publication-title: Energy – volume: 269 year: 2023 ident: b25 article-title: Double deep Q-network guided energy management strategy of a novel electric-hydraulic hybrid electric vehicle publication-title: Energy – volume: 8 year: 2007 ident: b48 article-title: Preventing over-fitting during model selection via Bayesian regularisation of the hyper-parameters publication-title: J Mach Learn Res – volume: 257 year: 2022 ident: b35 article-title: Learning-based supervisory control of dual mode engine-based hybrid electric vehicle with reliance on multivariate trip information publication-title: Energy Convers Manage – volume: 4 start-page: 372 year: 2021 end-page: 404 ident: b2 article-title: A review on electric vehicles: Technologies and challenges publication-title: Smart Cities – volume: 266 year: 2023 ident: b22 article-title: A comparative study of 13 deep reinforcement learning based energy management methods for a hybrid electric vehicle publication-title: Energy – volume: 342 year: 2023 ident: b24 article-title: An apprenticeship-reinforcement learning scheme based on expert demonstrations for energy management strategy of hybrid electric vehicles publication-title: Appl Energy – year: 2023 ident: b28 article-title: Hierarchical rewarding deep deterministic policy gradient strategy for energy management of hybrid electric vehicles publication-title: IEEE Trans Transp Electrif – volume: 14 start-page: 702 year: 2020 end-page: 711 ident: b32 article-title: Efficient energy management strategy for hybrid electric vehicles/plug-in hybrid electric vehicles: review and recent advances under intelligent transportation system publication-title: IET Intell Transp Syst – volume: 47 start-page: 7932 year: 2022 end-page: 7948 ident: b11 article-title: Optimized rule-based energy management for a polymer electrolyte membrane fuel cell/battery hybrid power system using a genetic algorithm publication-title: Int J Hydrog Energy – volume: 317 year: 2024 ident: b20 article-title: Integrated battery thermal and energy management for electric vehicles with hybrid energy storage system: A hierarchical approach publication-title: Energy Convers Manage – volume: 262 year: 2023 ident: b31 article-title: The application of machine learning based energy management strategy in multi-mode plug-in hybrid electric vehicle, part I: Twin Delayed Deep Deterministic Policy Gradient algorithm design for hybrid mode publication-title: Energy – start-page: 1625 year: 2009 ident: 10.1016/j.apenergy.2024.125171_b13 article-title: A generic dynamic programming matlab function – year: 2015 ident: 10.1016/j.apenergy.2024.125171_b26 – year: 2023 ident: 10.1016/j.apenergy.2024.125171_b28 article-title: Hierarchical rewarding deep deterministic policy gradient strategy for energy management of hybrid electric vehicles publication-title: IEEE Trans Transp Electrif – volume: 185 start-page: 1086 year: 2019 ident: 10.1016/j.apenergy.2024.125171_b12 article-title: Optimal rule design methodology for energy management strategy of a power-split hybrid electric bus publication-title: Energy doi: 10.1016/j.energy.2019.07.126 – volume: 317 year: 2024 ident: 10.1016/j.apenergy.2024.125171_b20 article-title: Integrated battery thermal and energy management for electric vehicles with hybrid energy storage system: A hierarchical approach publication-title: Energy Convers Manage doi: 10.1016/j.enconman.2024.118853 – volume: 269 year: 2023 ident: 10.1016/j.apenergy.2024.125171_b25 article-title: Double deep Q-network guided energy management strategy of a novel electric-hydraulic hybrid electric vehicle publication-title: Energy doi: 10.1016/j.energy.2023.126858 – start-page: 134 year: 2019 ident: 10.1016/j.apenergy.2024.125171_b37 article-title: Hyperparameter optimization for deep reinforcement learning in vehicle energy management – volume: 8 start-page: 6035 year: 2022 ident: 10.1016/j.apenergy.2024.125171_b21 article-title: Optimization-based energy management strategies for hybrid construction machinery: A review publication-title: Energy Rep doi: 10.1016/j.egyr.2022.04.050 – volume: 415 start-page: 295 year: 2020 ident: 10.1016/j.apenergy.2024.125171_b38 article-title: On hyperparameter optimization of machine learning algorithms: Theory and practice publication-title: Neurocomputing doi: 10.1016/j.neucom.2020.07.061 – volume: 16 start-page: 74 issue: 1 year: 2022 ident: 10.1016/j.apenergy.2024.125171_b10 article-title: Multi-objective energy management strategy for hybrid electric vehicles based on TD3 with non-parametric reward function publication-title: Energies doi: 10.3390/en16010074 – volume: 4 start-page: 1 issue: 1 year: 2021 ident: 10.1016/j.apenergy.2024.125171_b40 article-title: Efficient hyperparameter optimization for physics-based character animation publication-title: Proc ACM Comput Graph Interact Tech doi: 10.1145/3451254 – volume: 266 year: 2023 ident: 10.1016/j.apenergy.2024.125171_b22 article-title: A comparative study of 13 deep reinforcement learning based energy management methods for a hybrid electric vehicle publication-title: Energy doi: 10.1016/j.energy.2022.126497 – year: 2021 ident: 10.1016/j.apenergy.2024.125171_b42 – volume: 7 start-page: 1990 issue: 3 year: 2021 ident: 10.1016/j.apenergy.2024.125171_b6 article-title: Powertrain design and control in electrified vehicles: A critical review publication-title: IEEE Trans Transp Electrif doi: 10.1109/TTE.2021.3056432 – volume: 262 year: 2023 ident: 10.1016/j.apenergy.2024.125171_b31 article-title: The application of machine learning based energy management strategy in multi-mode plug-in hybrid electric vehicle, part I: Twin Delayed Deep Deterministic Policy Gradient algorithm design for hybrid mode publication-title: Energy doi: 10.1016/j.energy.2022.125084 – volume: Vol. 4 start-page: 2076 year: 2002 ident: 10.1016/j.apenergy.2024.125171_b18 article-title: Equivalent consumption minimization strategy for parallel hybrid powertrains – volume: 293 year: 2023 ident: 10.1016/j.apenergy.2024.125171_b47 article-title: Incentive learning-based energy management for hybrid energy storage system in electric vehicles publication-title: Energy Convers Manage doi: 10.1016/j.enconman.2023.117480 – volume: 236 start-page: 893 year: 2019 ident: 10.1016/j.apenergy.2024.125171_b17 article-title: Pontryagin’s minimum principle based model predictive control of energy management for a plug-in hybrid electric bus publication-title: Appl Energy doi: 10.1016/j.apenergy.2018.12.032 – volume: 224 year: 2021 ident: 10.1016/j.apenergy.2024.125171_b45 article-title: A novel energy management strategy of hybrid electric vehicle via an improved TD3 deep reinforcement learning publication-title: Energy doi: 10.1016/j.energy.2021.120118 – volume: 14 start-page: 3948 issue: 1 year: 2024 ident: 10.1016/j.apenergy.2024.125171_b41 article-title: The impact of Bayesian optimization on feature selection publication-title: Sci Rep doi: 10.1038/s41598-024-54515-w – volume: 248 start-page: 416 year: 2014 ident: 10.1016/j.apenergy.2024.125171_b15 article-title: Energy management of a power-split plug-in hybrid electric vehicle based on genetic algorithm and quadratic programming publication-title: J Power Sources doi: 10.1016/j.jpowsour.2013.09.085 – year: 2019 ident: 10.1016/j.apenergy.2024.125171_b3 – volume: 8 year: 2021 ident: 10.1016/j.apenergy.2024.125171_b4 article-title: Impacts of reducing air pollutants and CO2 emissions in urban road transport through 2035 in Chongqing, China publication-title: Environ Sci Ecotechnol doi: 10.1016/j.ese.2021.100125 – volume: 15 start-page: 50 issue: 1 year: 2020 ident: 10.1016/j.apenergy.2024.125171_b7 article-title: Research on synchronous control strategy of steer-by-wire system with dual steering actuator motors publication-title: Int J Veh Auton Syst doi: 10.1504/IJVAS.2020.104818 – year: 2023 ident: 10.1016/j.apenergy.2024.125171_b9 article-title: Energy management strategies for fuel cell vehicles: a comprehensive review of the latest progress in modeling, strategies, and future prospects publication-title: IEEE Trans Intell Transp Syst – ident: 10.1016/j.apenergy.2024.125171_b49 – year: 2022 ident: 10.1016/j.apenergy.2024.125171_b5 – year: 2018 ident: 10.1016/j.apenergy.2024.125171_b8 – volume: 123 year: 2023 ident: 10.1016/j.apenergy.2024.125171_b27 article-title: A deep reinforcement learning approach to energy management control with connected information for hybrid electric vehicles publication-title: Eng Appl Artif Intell doi: 10.1016/j.engappai.2023.106239 – volume: 257 year: 2022 ident: 10.1016/j.apenergy.2024.125171_b35 article-title: Learning-based supervisory control of dual mode engine-based hybrid electric vehicle with reliance on multivariate trip information publication-title: Energy Convers Manage doi: 10.1016/j.enconman.2022.115450 – volume: 342 year: 2023 ident: 10.1016/j.apenergy.2024.125171_b24 article-title: An apprenticeship-reinforcement learning scheme based on expert demonstrations for energy management strategy of hybrid electric vehicles publication-title: Appl Energy doi: 10.1016/j.apenergy.2023.121227 – volume: 10 issue: 10 year: 2022 ident: 10.1016/j.apenergy.2024.125171_b34 article-title: Energy management in plugin hybrid electric vehicles with hybrid energy storage system using hybrid approach publication-title: Energy Technol doi: 10.1002/ente.202200355 – year: 2022 ident: 10.1016/j.apenergy.2024.125171_b43 – start-page: 32 year: 2020 ident: 10.1016/j.apenergy.2024.125171_b36 article-title: Deep reinforcement learning using cyclical learning rates – volume: 8 issue: 4 year: 2007 ident: 10.1016/j.apenergy.2024.125171_b48 article-title: Preventing over-fitting during model selection via Bayesian regularisation of the hyper-parameters publication-title: J Mach Learn Res – start-page: 1 year: 2019 ident: 10.1016/j.apenergy.2024.125171_b39 article-title: Simultaneous electric powertrain hardware and energy management optimization of a hybrid electric vehicle using deep reinforcement learning and Bayesian optimization – volume: 273 year: 2023 ident: 10.1016/j.apenergy.2024.125171_b14 article-title: Fast dynamic-programming algorithm for solving global optimization problems of hybrid electric vehicles publication-title: Energy doi: 10.1016/j.energy.2023.127207 – volume: 1 start-page: 99 issue: 1 year: 2020 ident: 10.1016/j.apenergy.2024.125171_b1 article-title: Future material demand for automotive lithium-based batteries publication-title: Commun Mater doi: 10.1038/s43246-020-00095-x – volume: 7 start-page: 616 issue: 2 year: 2020 ident: 10.1016/j.apenergy.2024.125171_b16 article-title: Modified particle swarm optimization with chaotic attraction strategy for modular design of hybrid powertrains publication-title: IEEE Trans Transp Electrif doi: 10.1109/TTE.2020.3014688 – start-page: 1 year: 2021 ident: 10.1016/j.apenergy.2024.125171_b33 article-title: Deep reinforcement learning based energy management strategy for fuel cell and battery powered rail vehicles – volume: 8 start-page: 2516 issue: 2 year: 2022 ident: 10.1016/j.apenergy.2024.125171_b19 article-title: An optimal control strategy for plug-in hybrid electric vehicles based on enhanced model predictive control with efficient numerical method publication-title: IEEE Trans Transp Electrif doi: 10.1109/TTE.2022.3141191 – year: 2023 ident: 10.1016/j.apenergy.2024.125171_b23 – year: 2022 ident: 10.1016/j.apenergy.2024.125171_b44 article-title: High robustness energy management strategy of hybrid electric vehicle based on improved soft actor-critic deep reinforcement learning publication-title: Energy doi: 10.1016/j.energy.2022.124806 – start-page: 1587 year: 2018 ident: 10.1016/j.apenergy.2024.125171_b29 article-title: Addressing function approximation error in actor-critic methods – volume: 348 year: 2023 ident: 10.1016/j.apenergy.2024.125171_b30 article-title: Energy management of multi-mode plug-in hybrid electric vehicle using multi-agent deep reinforcement learning publication-title: Appl Energy doi: 10.1016/j.apenergy.2023.121526 – volume: 47 start-page: 7932 issue: 12 year: 2022 ident: 10.1016/j.apenergy.2024.125171_b11 article-title: Optimized rule-based energy management for a polymer electrolyte membrane fuel cell/battery hybrid power system using a genetic algorithm publication-title: Int J Hydrog Energy doi: 10.1016/j.ijhydene.2021.12.121 – volume: 4 start-page: 372 issue: 1 year: 2021 ident: 10.1016/j.apenergy.2024.125171_b2 article-title: A review on electric vehicles: Technologies and challenges publication-title: Smart Cities doi: 10.3390/smartcities4010022 – volume: 14 start-page: 702 issue: 7 year: 2020 ident: 10.1016/j.apenergy.2024.125171_b32 article-title: Efficient energy management strategy for hybrid electric vehicles/plug-in hybrid electric vehicles: review and recent advances under intelligent transportation system publication-title: IET Intell Transp Syst doi: 10.1049/iet-its.2019.0606 – volume: 70 start-page: 9922 issue: 10 year: 2021 ident: 10.1016/j.apenergy.2024.125171_b46 article-title: Distributed deep reinforcement learning-based energy and emission management strategy for hybrid electric vehicles publication-title: IEEE Trans Veh Technol doi: 10.1109/TVT.2021.3107734 |
| SSID | ssj0002120 |
| Score | 2.5093098 |
| Snippet | Hybridization and electrification of vehicles are underway to achieve Net-zero emissions for road transport. The upcoming deep reinforcement learning (DRL)... |
| SourceID | proquest crossref elsevier |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 125171 |
| SubjectTerms | algorithms Bayesian optimization Bayesian theory brittleness energy Energy management strategy hybridization issues and policy Non-parametric reward function Plug-in hybrid electric vehicles road transportation Twin Delayed Deep Deterministic Policy Gradients |
| Title | Bayesian optimization for hyper-parameter tuning of an improved twin delayed deep deterministic policy gradients based energy management strategy for plug-in hybrid electric vehicles |
| URI | https://dx.doi.org/10.1016/j.apenergy.2024.125171 https://www.proquest.com/docview/3165865938 |
| Volume | 381 |
| WOSCitedRecordID | wos001398530400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 issn: 0306-2619 databaseCode: AIEXJ dateStart: 19950101 customDbUrl: isFulltext: true dateEnd: 99991231 titleUrlDefault: https://www.sciencedirect.com omitProxy: false ssIdentifier: ssj0002120 providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1Lb9NAEF6FlAMcEBSqlpcWiZvlUK-feywoFSBUcSgoN2uzXiepEsckdtr8MX4Cv4vZl-2GR-mBixNZ3rGt-bLz7eabGYRex5TynHHqZkRkbhCEvptEPHRjQSAgMcF81bTv66f47CwZjejnXu-HzYXZzOOiSK6uaPlfXQ3nwNkydfYW7m6Mwgn4Dk6HI7gdjv_k-LdsK1Rm5BJmg4VJs1RqwimsOVeuLPa9kCIYp6oLo3mGq2dqdwHoZ3WpBLJzMJPBpyjhoCUzqqazauvAt85kpcRi1dqRgTBzhE4iXDRyGmetC99qRWg5rycuGJ5uZYqYo7vvgLWNmCphXpckW2asTbZb_kY8PCumbNYScCMaKCbfbBSWk5je1z2tL-sWuEznCRTdnQ4StlIvm-F1HLlyxdedvX3d8cXMv5Ku6ZYuv4QGvUtxMWClfvoB3CIYtAOu1-LeiZGNctGK4i5SayeVdlJt5w7aI3FIkz7aO_kwHH1sOAExBULtG3Ry1X__RH-iSTuEQbGg84fogVm-4BMNu0eoJ4p9dL9T1HIfHQzb3Em41ASP9WP03SITd5GJAR14B5lYIxMvcwxXW2RiiUxskIklMvE1ZGKNTNwgEytkYv3auEUmtshU9zbIxBqZ2CITW2Q-QV9Oh-fv3rumbYjL_SCsXFlhkZBEkGjssTAjAY-IyOMs8zgNgoCL45iHPAuj3M9gscDGguVRNgaiLoCo-WP_APWLZSEOEc4p4R5E4CCiNGA5pcIjhHlZSP0xE7E4QqH1UspNTX3Z2mWe_h0nR-hNM67UVWVuHEEtCFLDjTXnTQHfN459ZVGTQvCQ_wiyQizrdep7sACJ4G2Sp7d-omfoXvsjfY761aoWL9Bdvqlm69VL8wP4CeqN9ys |
| linkProvider | Elsevier |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Bayesian+optimization+for+hyper-parameter+tuning+of+an+improved+twin+delayed+deep+deterministic+policy+gradients+based+energy+management+strategy+for+plug-in+hybrid+electric+vehicles&rft.jtitle=Applied+energy&rft.au=Wang%2C+Jinhai&rft.au=Du%2C+Changqing&rft.au=Yan%2C+Fuwu&rft.au=Hua%2C+Min&rft.date=2025-03-01&rft.issn=0306-2619&rft.volume=381&rft.spage=125171&rft_id=info:doi/10.1016%2Fj.apenergy.2024.125171&rft.externalDBID=n%2Fa&rft.externalDocID=10_1016_j_apenergy_2024_125171 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0306-2619&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0306-2619&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0306-2619&client=summon |