Battery Degradation Oriented Active Control Strategy by Using a Reinforcement Learning Algorithm in Hybrid Energy Storage System

The integration of ultracapacitors (UCs) into hybrid energy storage systems is a solution to mitigate battery degradation. Traditional strategies focus on fuel cell and battery power regulation while treating UC management as a passive element, resulting in suboptimal UC utilization. To optimize the...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on industrial electronics (1982) Vol. 72; no. 5; pp. 4922 - 4932
Main Authors: Lin, Xinyou, Huang, Hao, Xu, Xinhao
Format: Journal Article
Language:English
Published: New York IEEE 01.05.2025
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Active control
Algorithms
Batteries
Control systems
Deep reinforcement learning (RL)
Degradation
Electric charge
Energy management
Energy storage
Energy utilization
fuel cell hybrid electric vehicle (FCHEV)
Fuel cells
hybrid energy storage system (HESS)
Hybrid systems
Machine learning
Numerical models
Optimization
Power generation
Real time
Real-time systems
State of charge
ISSN:0278-0046, 1557-9948
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract The integration of ultracapacitors (UCs) into hybrid energy storage systems is a solution to mitigate battery degradation. Traditional strategies focus on fuel cell and battery power regulation while treating UC management as a passive element, resulting in suboptimal UC utilization. To optimize the energy utilization of UCs, this article proposes an active state control strategy within the hybrid system. Initially, leveraging the battery severity factor, the optimal power split strategy for HESS is proposed for a reference state-of-charge (SOC) of UC. Subsequently, a driving pattern severity factor is designed, and an online self-learning Markov predictor is employed to quantify the operational state of vehicle. To provide optimal reference SOC guidance to HESS in real time, a reinforcement learning algorithm featuring an experience replay mechanism is developed. Utilizing pretrained agents that integrate vehicle driving state abstraction parameters, the system generates the reference SOC of UC, enabling the optimal battery-UC power split in real time. Both numerical and semiphysical validations confirm the efficacy of the proposed strategy in enhancing the power output ratio of UC, optimizing energy storage space utilization, and reducing the battery severity factor, consequently improving overall battery lifespan.
AbstractList The integration of ultracapacitors (UCs) into hybrid energy storage systems is a solution to mitigate battery degradation. Traditional strategies focus on fuel cell and battery power regulation while treating UC management as a passive element, resulting in suboptimal UC utilization. To optimize the energy utilization of UCs, this article proposes an active state control strategy within the hybrid system. Initially, leveraging the battery severity factor, the optimal power split strategy for HESS is proposed for a reference state-of-charge (SOC) of UC. Subsequently, a driving pattern severity factor is designed, and an online self-learning Markov predictor is employed to quantify the operational state of vehicle. To provide optimal reference SOC guidance to HESS in real time, a reinforcement learning algorithm featuring an experience replay mechanism is developed. Utilizing pretrained agents that integrate vehicle driving state abstraction parameters, the system generates the reference SOC of UC, enabling the optimal battery-UC power split in real time. Both numerical and semiphysical validations confirm the efficacy of the proposed strategy in enhancing the power output ratio of UC, optimizing energy storage space utilization, and reducing the battery severity factor, consequently improving overall battery lifespan.
Author Xu, Xinhao
Huang, Hao
Lin, Xinyou
Author_xml – sequence: 1
  givenname: Xinyou
  orcidid: 0000-0002-4061-1055
  surname: Lin
  fullname: Lin, Xinyou
  email: linxinyoou@fzu.edu.cn
  organization: School of Mechanical Engineering and Auto Mobile, Fuzhou University, Fuzhou, China
– sequence: 2
  givenname: Hao
  orcidid: 0009-0007-7228-4208
  surname: Huang
  fullname: Huang, Hao
  email: 230220018@fzu.edu.cn
  organization: School of Mechanical Engineering and Auto Mobile, Fuzhou University, Fuzhou, China
– sequence: 3
  givenname: Xinhao
  orcidid: 0009-0002-4107-8026
  surname: Xu
  fullname: Xu, Xinhao
  email: xinhaoxu@tesla.com
  organization: School of Mechanical Engineering and Auto Mobile, Fuzhou University, Fuzhou, China
BookMark eNpNkDFvwjAQRq2KSgXavUMHS51DbcdJnJFSWpCQkArMkZNcUiNi04uplK0_vUEwdLrhvved7o3IwDoLhDxyNuGcpS_b5XwimJCTUCquFLshQx5FSZCmUg3IkIlEBYzJ-I6M2nbPGJcRj4bk91V7D9jRN6hRl9obZ-kaDVgPJZ0W3vwAnTnr0R3oxqP2UHc07-iuNbammn6CsZXDApoeoSvQaM-L6aF2aPxXQ42liy5HU9K5BezhjXeoa6CbrvXQ3JPbSh9aeLjOMdm9z7ezRbBafyxn01VQCBn5QLEqjHSpmI6SuEi4znmSJ6LgoArVfyzjKg5LUYo01zHPZSpFXGiuq7LKdSohHJPnS-8R3fcJWp_t3QltfzILe19c8FCKPsUuqQJd2yJU2RFNo7HLOMvOnrPec3b2nF0998jTBTEA8C-eSBaqNPwD1e181w
CODEN ITIED6
Cites_doi 10.1016/j.ijepes.2012.06.026
10.1109/TIE.2015.2475419
10.1016/j.epsr.2023.109235
10.1109/TITS.2022.3233564
10.1016/j.apenergy.2019.113707
10.1109/TMECH.2018.2832019
10.1109/TIE.2020.2977574
10.1016/j.jpowsour.2019.04.083
10.1109/TTE.2015.2471180
10.1109/TPEL.2020.3028154
10.1109/TMECH.2017.2707338
10.1016/j.enconman.2022.116383
10.1016/j.energy.2023.129128
10.1016/j.jpowsour.2020.227964
10.1109/TIE.2020.2979528
10.1109/TVT.2022.3168870
10.1016/j.ijhydene.2020.01.017
10.1016/j.energy.2024.130728
10.1016/j.jpowsour.2018.05.041
10.1109/TVT.2020.3028089
10.1109/TIE.2021.3062273
10.1109/TIE.2018.2850030
10.1016/j.energy.2022.125620
10.1016/j.energy.2023.129173
10.1016/j.energy.2021.121593
ContentType Journal Article
Copyright Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025
Copyright_xml – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025
DBID 97E
RIA
RIE
AAYXX
CITATION
7SP
8FD
L7M
DOI 10.1109/TIE.2024.3481880
DatabaseName IEEE All-Society Periodicals Package (ASPP) 2005–Present
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE/IET Electronic Library (IEL) (UW System Shared)
CrossRef
Electronics & Communications Abstracts
Technology Research Database
Advanced Technologies Database with Aerospace
DatabaseTitle CrossRef
Technology Research Database
Advanced Technologies Database with Aerospace
Electronics & Communications Abstracts
DatabaseTitleList Technology Research Database
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE/IET Electronic Library (IEL) (UW System Shared)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
EISSN 1557-9948
EndPage 4932
ExternalDocumentID 10_1109_TIE_2024_3481880
10740389
Genre orig-research
GrantInformation_xml – fundername: National Natural Science Foundation of China
  grantid: 52272389
GroupedDBID -~X
.DC
0R~
29I
4.4
5GY
5VS
6IK
97E
9M8
AAJGR
AARMG
AASAJ
AAWTH
ABAZT
ABQJQ
ABVLG
ACGFO
ACGFS
ACIWK
ACKIV
ACNCT
AENEX
AETIX
AGQYO
AGSQL
AHBIQ
AI.
AIBXA
AKJIK
AKQYR
ALLEH
ALMA_UNASSIGNED_HOLDINGS
ASUFR
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CS3
DU5
EBS
EJD
HZ~
H~9
IBMZZ
ICLAB
IFIPE
IFJZH
IPLJI
JAVBF
LAI
M43
MS~
O9-
OCL
P2P
RIA
RIE
RNS
TAE
TN5
TWZ
VH1
VJK
AAYXX
CITATION
7SP
8FD
L7M
ID FETCH-LOGICAL-c245t-80f35ad80a576c71ab17b72c1e8c848146f63d2d29ba61b49426ca1afdfba94e3
IEDL.DBID RIE
ISICitedReferencesCount 1
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001346681900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 0278-0046
IngestDate Tue Jul 22 16:40:56 EDT 2025
Sat Nov 29 08:04:56 EST 2025
Wed Aug 27 02:04:40 EDT 2025
IsPeerReviewed true
IsScholarly true
Issue 5
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
https://doi.org/10.15223/policy-029
https://doi.org/10.15223/policy-037
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c245t-80f35ad80a576c71ab17b72c1e8c848146f63d2d29ba61b49426ca1afdfba94e3
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0009-0002-4107-8026
0009-0007-7228-4208
0000-0002-4061-1055
PQID 3188121342
PQPubID 85464
PageCount 11
ParticipantIDs proquest_journals_3188121342
ieee_primary_10740389
crossref_primary_10_1109_TIE_2024_3481880
PublicationCentury 2000
PublicationDate 2025-05-01
PublicationDateYYYYMMDD 2025-05-01
PublicationDate_xml – month: 05
  year: 2025
  text: 2025-05-01
  day: 01
PublicationDecade 2020
PublicationPlace New York
PublicationPlace_xml – name: New York
PublicationTitle IEEE transactions on industrial electronics (1982)
PublicationTitleAbbrev TIE
PublicationYear 2025
Publisher IEEE
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml – name: IEEE
– name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References ref13
ref12
ref15
ref14
ref11
ref10
ref2
Sheng (ref6) 2022; 47
ref1
ref17
ref16
ref19
ref18
ref24
ref23
ref26
ref25
ref20
ref22
ref21
ref8
ref7
ref9
ref4
ref3
ref5
References_xml – ident: ref10
  doi: 10.1016/j.ijepes.2012.06.026
– ident: ref17
  doi: 10.1109/TIE.2015.2475419
– ident: ref12
  doi: 10.1016/j.epsr.2023.109235
– ident: ref23
  doi: 10.1109/TITS.2022.3233564
– ident: ref24
  doi: 10.1016/j.apenergy.2019.113707
– ident: ref9
  doi: 10.1109/TMECH.2018.2832019
– ident: ref1
  doi: 10.1109/TIE.2020.2977574
– ident: ref7
  doi: 10.1016/j.jpowsour.2019.04.083
– ident: ref25
  doi: 10.1109/TTE.2015.2471180
– ident: ref4
  doi: 10.1109/TPEL.2020.3028154
– ident: ref18
  doi: 10.1109/TMECH.2017.2707338
– ident: ref15
  doi: 10.1016/j.enconman.2022.116383
– ident: ref16
  doi: 10.1016/j.energy.2023.129128
– ident: ref20
  doi: 10.1016/j.jpowsour.2020.227964
– ident: ref2
  doi: 10.1109/TIE.2020.2979528
– ident: ref21
  doi: 10.1109/TVT.2022.3168870
– ident: ref5
  doi: 10.1016/j.ijhydene.2020.01.017
– ident: ref26
  doi: 10.1016/j.energy.2024.130728
– volume: 47
  year: 2022
  ident: ref6
  article-title: Experimental investigation of supercapacitor based regenerative energy storage for a fuel cell vehicle equipped with an alternator
  publication-title: Int. J. Hydrogen Energy
– ident: ref13
  doi: 10.1016/j.jpowsour.2018.05.041
– ident: ref14
  doi: 10.1109/TVT.2020.3028089
– ident: ref22
  doi: 10.1109/TIE.2021.3062273
– ident: ref3
  doi: 10.1109/TIE.2018.2850030
– ident: ref8
  doi: 10.1016/j.energy.2022.125620
– ident: ref11
  doi: 10.1016/j.energy.2023.129173
– ident: ref19
  doi: 10.1016/j.energy.2021.121593
SSID ssj0014515
Score 2.4813442
Snippet The integration of ultracapacitors (UCs) into hybrid energy storage systems is a solution to mitigate battery degradation. Traditional strategies focus on fuel...
SourceID proquest
crossref
ieee
SourceType Aggregation Database
Index Database
Publisher
StartPage 4922
SubjectTerms Active control
Algorithms
Batteries
Control systems
Deep reinforcement learning (RL)
Degradation
Electric charge
Energy management
Energy storage
Energy utilization
fuel cell hybrid electric vehicle (FCHEV)
Fuel cells
hybrid energy storage system (HESS)
Hybrid systems
Machine learning
Numerical models
Optimization
Power generation
Real time
Real-time systems
State of charge
Title Battery Degradation Oriented Active Control Strategy by Using a Reinforcement Learning Algorithm in Hybrid Energy Storage System
URI https://ieeexplore.ieee.org/document/10740389
https://www.proquest.com/docview/3188121342
Volume 72
WOSCitedRecordID wos001346681900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVIEE
  databaseName: IEEE/IET Electronic Library (IEL) (UW System Shared)
  customDbUrl:
  eissn: 1557-9948
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0014515
  issn: 0278-0046
  databaseCode: RIE
  dateStart: 19820101
  isFulltext: true
  titleUrlDefault: https://ieeexplore.ieee.org/
  providerName: IEEE
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlZ3NS8MwFMCDGx704OfE6ZR38OKhW5tmbXIcc0NBpsiU3UqSpnOgnWyd0Jt_uvnoZCAevPXQlpL3-j6S934PoSvCaZZ1sVZeEqWe8XieiGXoaV9vhqhyziyO4eU-Ho3oZMIeq2Z12wujlLLFZ6ptLu1ZfjqXK7NV1jHFgwYIV0O1OI5cs9bPkQHpunEF2CBjdda3PpP0WWd8N9CZICZt03VKDQFywwfZoSq_LLF1L8P9f37YAdqr4kjoOcEfoi2VH6HdDbrgMfpy7MwSbgwQws1OggfDNdZRJvSsoYO-K1WHilJbgijBlhEAhydlsarS7iBCRWKdQu9tOl_Mitd3mOVwW5qWLxjYFkL9Fq1RUwWOg95Az8PBuH_rVQMXPIlJt9DeKgu7PKU-11mIjAMugljEWAaKSoPdJ1EWhSlOMRM8CgRh2r1LHvAszQRnRIUnqJ7Pc3WKgAaKCV9yFQlChNRRUUBE7HPJJZV-xproei2C5MNxNRKbj_gs0eJKjLiSSlxN1DBLvnGfW-0maq2FllR_3jLRNopaTB0---Oxc7SDzRBfW7XYQvVisVIXaFt-FrPl4tIq1TcxFsvI
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlZ1LS8NAEIAHrYJ68FmxPufgxUNqst089li0pWKtIlW8hd3Npha0lT6E3vzp7iOVgnjwlkMSws5kHrsz3wCcU57keUi08tIo84zH80Qsa5729WaIKufM4hie23Gnk7y8sIeiWd32wiilbPGZqppLe5afDeXUbJVdmuJBA4RbhpWQUuK7dq2fQwMauoEFxEBjdd43P5X02WX3pqFzQUKrpu80MQzIBS9kx6r8ssXWwTS3_vlp27BZRJJYd6LfgSU12IWNBb7gHnw5euYMrw0Swk1PwntDNtZxJtatqcMrV6yOBad2hmKGtpAAOT4qC1aVdg8RCxZrD-tvveGoP3l9x_4AWzPT9IUN20So36J1qqfQkdDL8NRsdK9aXjFywZOEhhPtr_JayLPE5zoPkXHARRCLmMhAJdKA92mUR7WMZIQJHgWCMu3gJQ94nuWCM6pq-1AaDAfqADAJFBO-5CoSlAqp46KAitjnkstE-jmrwMVcBOmHI2ukNiPxWarFlRpxpYW4KlA2S75wn1vtChzPhZYW_9441VYqsaA6cvjHY2ew1uretdP2Tef2CNaJGelraxiPoTQZTdUJrMrPSX88OrUK9g3xE88P
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Battery+Degradation+Oriented+Active+Control+Strategy+by+Using+a+Reinforcement+Learning+Algorithm+in+Hybrid+Energy+Storage+System&rft.jtitle=IEEE+transactions+on+industrial+electronics+%281982%29&rft.au=Lin%2C+Xinyou&rft.au=Huang%2C+Hao&rft.au=Xu%2C+Xinhao&rft.date=2025-05-01&rft.pub=IEEE&rft.issn=0278-0046&rft.volume=72&rft.issue=5&rft.spage=4922&rft.epage=4932&rft_id=info:doi/10.1109%2FTIE.2024.3481880&rft.externalDocID=10740389
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0278-0046&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0278-0046&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0278-0046&client=summon