Multi-Scale Reinforcement Learning of Dynamic Energy Controller for Connected Electrified Vehicles
The synergy of reinforcement learning (RL)-based energy management and vehicle-to-everything communication has been proved effective in boosting the fuel economy of connected plug-in hybrid electric vehicles (PHEVs). However, the intricate coupling of mechanical, electrical, thermal states and drivi...
Uložené v:
| Vydané v: | IEEE transactions on intelligent transportation systems s. 1 - 13 |
|---|---|
| Hlavní autori: | , , , , |
| Médium: | Journal Article |
| Jazyk: | English |
| Vydavateľské údaje: |
IEEE
2025
|
| Predmet: | |
| ISSN: | 1524-9050, 1558-0016 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | The synergy of reinforcement learning (RL)-based energy management and vehicle-to-everything communication has been proved effective in boosting the fuel economy of connected plug-in hybrid electric vehicles (PHEVs). However, the intricate coupling of mechanical, electrical, thermal states and driving cycle results in a high-dimensional complex energy control problem for PHEVs, which is challenging to optimally solve within the same time scale. To this end, this study designs a multi-horizon reinforcement learning (MHRL)-based energy management of PHEVs, aware of the traffic preview from intelligent transportation systems to optimize the energy flow and thermal states as well as the transient dynamics of the powertrain. The proposed strategy features a novel state space representation, and solves the coordinated training among multiple sub-networks belonging to different control tasks in various time scales. Simulation and hardware-in-the-loop experiments are carried out based on a standard driving cycle and a real-world driving cycle with real-time traffic data demonstrate that the MHRL strategy improves fuel economy by 3.0%~7.9% compared to conventional RL-based energy management under various coolant temperature conditions and dynamic driving scenarios. |
|---|---|
| AbstractList | The synergy of reinforcement learning (RL)-based energy management and vehicle-to-everything communication has been proved effective in boosting the fuel economy of connected plug-in hybrid electric vehicles (PHEVs). However, the intricate coupling of mechanical, electrical, thermal states and driving cycle results in a high-dimensional complex energy control problem for PHEVs, which is challenging to optimally solve within the same time scale. To this end, this study designs a multi-horizon reinforcement learning (MHRL)-based energy management of PHEVs, aware of the traffic preview from intelligent transportation systems to optimize the energy flow and thermal states as well as the transient dynamics of the powertrain. The proposed strategy features a novel state space representation, and solves the coordinated training among multiple sub-networks belonging to different control tasks in various time scales. Simulation and hardware-in-the-loop experiments are carried out based on a standard driving cycle and a real-world driving cycle with real-time traffic data demonstrate that the MHRL strategy improves fuel economy by 3.0%~7.9% compared to conventional RL-based energy management under various coolant temperature conditions and dynamic driving scenarios. |
| Author | Zhang, Junzhi Lei, Nuo Zhang, Hao Wang, Zhi Li, Shengbo Eben |
| Author_xml | – sequence: 1 givenname: Hao orcidid: 0000-0001-5395-4017 surname: Zhang fullname: Zhang, Hao email: hao_thu@foxmail.com organization: School of Vehicle and Mobility, State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University, Beijing, China – sequence: 2 givenname: Nuo orcidid: 0000-0001-8731-7497 surname: Lei fullname: Lei, Nuo email: lein21@mails.tsinghua.edu.cn organization: School of Vehicle and Mobility, State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University, Beijing, China – sequence: 3 givenname: Shengbo Eben orcidid: 0000-0003-4923-3633 surname: Li fullname: Li, Shengbo Eben email: lein21@mails.tsinghua.edu.cn organization: School of Vehicle and Mobility, State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University, Beijing, China – sequence: 4 givenname: Junzhi orcidid: 0000-0002-5055-2941 surname: Zhang fullname: Zhang, Junzhi email: jzhzhang@tsinghua.edu.cn organization: School of Vehicle and Mobility, State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University, Beijing, China – sequence: 5 givenname: Zhi orcidid: 0009-0006-3287-9007 surname: Wang fullname: Wang, Zhi email: wangzhi@tsinghua.edu.cn organization: School of Vehicle and Mobility, State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University, Beijing, China |
| BookMark | eNpFkM1KAzEUhYNUsK0-gOAiLzA1Nz_zs5RatVARbHU7ZJKbGkkzkhkXfXtnaMHVOfdwzl18MzKJbURCboEtAFh1v1vvtgvOuFqInPMKigsyBaXKjDHIJ6PnMquYYldk1nXfQyoVwJQ0r7-h99nW6ID0HX10bTJ4wNjTDeoUfdzT1tHHY9QHb-gqYtof6bKNfWpDwESH_nhGND1augqDJu_84D_xy5uA3TW5dDp0eHPWOfl4Wu2WL9nm7Xm9fNhkhnPWZ5bZvMybxhquK20LFM5CJSQiFE0pweVCFiWWOXeSc6GVszaXpZVCaQ7IxJzA6a9JbdcldPVP8gedjjWweoRUj5DqEVJ9hjRs7k4bj4j_feCgqqoUfyiiZps |
| CODEN | ITISFG |
| ContentType | Journal Article |
| DBID | 97E RIA RIE AAYXX CITATION |
| DOI | 10.1109/TITS.2025.3622917 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1558-0016 |
| EndPage | 13 |
| ExternalDocumentID | 10_1109_TITS_2025_3622917 11215998 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Shuimu Tsinghua Postdoctoral Fellowship – fundername: National Natural Science Foundation of China grantid: T2241003 funderid: 10.13039/501100001809 |
| GroupedDBID | -~X 0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK ACNCT AENEX AGQYO AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS HZ~ IFIPE IPLJI JAVBF LAI O9- OCL P2P PQQKQ RIA RIE RNS AAYXX AETIX AGSQL AIBXA CITATION EJD H~9 M43 ZY4 |
| ID | FETCH-LOGICAL-c220t-d0d686bbdc2a9ad7e3fd1934ee17b841f63478e862f4223a5fdd648d435a21e03 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 0 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001600829800001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1524-9050 |
| IngestDate | Sat Nov 29 07:02:33 EST 2025 Wed Oct 29 06:12:24 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c220t-d0d686bbdc2a9ad7e3fd1934ee17b841f63478e862f4223a5fdd648d435a21e03 |
| ORCID | 0000-0001-5395-4017 0000-0002-5055-2941 0000-0001-8731-7497 0009-0006-3287-9007 0000-0003-4923-3633 |
| PageCount | 13 |
| ParticipantIDs | crossref_primary_10_1109_TITS_2025_3622917 ieee_primary_11215998 |
| PublicationCentury | 2000 |
| PublicationDate | 2025-00-00 |
| PublicationDateYYYYMMDD | 2025-01-01 |
| PublicationDate_xml | – year: 2025 text: 2025-00-00 |
| PublicationDecade | 2020 |
| PublicationTitle | IEEE transactions on intelligent transportation systems |
| PublicationTitleAbbrev | TITS |
| PublicationYear | 2025 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0014511 |
| Score | 2.4315593 |
| Snippet | The synergy of reinforcement learning (RL)-based energy management and vehicle-to-everything communication has been proved effective in boosting the fuel... |
| SourceID | crossref ieee |
| SourceType | Index Database Publisher |
| StartPage | 1 |
| SubjectTerms | Autoencoders Data models Decoding Energy management Engines Generators Mechanical power transmission multi-horizon plug-in hybrid electric vehicles Reinforcement learning traffic preview Transient analysis Vectors Vehicle dynamics |
| Title | Multi-Scale Reinforcement Learning of Dynamic Energy Controller for Connected Electrified Vehicles |
| URI | https://ieeexplore.ieee.org/document/11215998 |
| WOSCitedRecordID | wos001600829800001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-0016 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014511 issn: 1524-9050 databaseCode: RIE dateStart: 20000101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV09T8MwELWgYoCBzyLKlzwwIbl1bMeOR1RawVIhWlC3KLHPUAm1qLT8fmwnpV0Y2KLEkaI7K_fufPceQjfah3H_RBKTFY4IQzkpdekIF1Qk2gIvwEaxCTUYZOOxfqqH1eMsDADE5jNoh8t4lm9nZhlKZZ0kUCH4_GAbbSslq2Gt3yODQLQVyVGZIJqmqyPMhOrO6HE09KkgS9v-d810FCdbB6ENVZUYVPoH__ycQ7Rfo0d8V7n7CG3B9BjtbXAKnqAyjtSSobc94GeIxKgm1gBxzaX6hmcO31dK9LgXZ_9wt-pY_4A59utxbH8xHoziXpTJmTiPVPErvMcmuiZ66fdG3QdSCykQwxhdEEutzGRZWsMKXVgF3FkP3ARAospMJE5yoTLwyY0THi4UqbNWisx6KFWwBCg_RY3pbApnCAsBlhfMmlSxIE6nU1YyzaRikieGuRa6XVk2_6z4MvKYZ1CdBzfkwQ157YYWagarrhfWBj3_4_4F2g2vVxWQS9RYzJdwhXbM92LyNb-O2-EHLKqz2Q |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LTwIxEG4UTdSDT4z47MGTSbHbdh89GoRARGIEDbfNbjtVEgMGwd9v212Eiwdvm91ms5lpdr6ZznwfQtfShnH7JCIqyQwRinKSy9wQLqgIpAaegfZiE3GvlwyH8qkcVvezMADgm8-g7i79Wb6eqLkrld0GjgrB5gfraMNJZ5XjWr-HBo5qy9OjMkEkDReHmAGVt4POoG-TQRbW7Q-bSS9PtgxDK7oqPqy09v75Qftot8SP-K5w-AFag_Eh2llhFTxCuR-qJX1rfcDP4KlRla8C4pJN9Q1PDL4vtOhx00__4UbRs_4BU2zXY98AoywcxU0vlDMyFqviV3j3bXRV9NJqDhptUkopEMUYnRFNdZREea4Vy2SmY-BGW-gmAII4T0RgIi7iBGx6Y4QFDFlotI5Eoi2YylgAlB-jyngyhhOEhQDNM6ZVGDMnTydDljPJophFPFDM1NDNwrLpZ8GYkfpMg8rUuSF1bkhLN9RQ1Vl1ubA06Okf96_QVnvw2E27nd7DGdp2ryrqIeeoMpvO4QJtqu_Z6Gt66bfGD7r1tyI |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Multi-Scale+Reinforcement+Learning+of+Dynamic+Energy+Controller+for+Connected+Electrified+Vehicles&rft.jtitle=IEEE+transactions+on+intelligent+transportation+systems&rft.au=Zhang%2C+Hao&rft.au=Lei%2C+Nuo&rft.au=Li%2C+Shengbo+Eben&rft.au=Zhang%2C+Junzhi&rft.date=2025&rft.pub=IEEE&rft.issn=1524-9050&rft.spage=1&rft.epage=13&rft_id=info:doi/10.1109%2FTITS.2025.3622917&rft.externalDocID=11215998 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1524-9050&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1524-9050&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1524-9050&client=summon |