Battery Degradation Oriented Active Control Strategy by Using a Reinforcement Learning Algorithm in Hybrid Energy Storage System

The integration of ultracapacitors (UCs) into hybrid energy storage systems is a solution to mitigate battery degradation. Traditional strategies focus on fuel cell and battery power regulation while treating UC management as a passive element, resulting in suboptimal UC utilization. To optimize the...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	IEEE transactions on industrial electronics (1982) Ročník 72; číslo 5; s. 4922 - 4932
Hlavní autori:	Lin, Xinyou, Huang, Hao, Xu, Xinhao
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	New York IEEE 01.05.2025 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:	Active control Algorithms Batteries Control systems Deep reinforcement learning (RL) Degradation Electric charge Energy management Energy storage Energy utilization fuel cell hybrid electric vehicle (FCHEV) Fuel cells hybrid energy storage system (HESS) Hybrid systems Machine learning Numerical models Optimization Power generation Real time Real-time systems State of charge
ISSN:	0278-0046, 1557-9948
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Abstract	The integration of ultracapacitors (UCs) into hybrid energy storage systems is a solution to mitigate battery degradation. Traditional strategies focus on fuel cell and battery power regulation while treating UC management as a passive element, resulting in suboptimal UC utilization. To optimize the energy utilization of UCs, this article proposes an active state control strategy within the hybrid system. Initially, leveraging the battery severity factor, the optimal power split strategy for HESS is proposed for a reference state-of-charge (SOC) of UC. Subsequently, a driving pattern severity factor is designed, and an online self-learning Markov predictor is employed to quantify the operational state of vehicle. To provide optimal reference SOC guidance to HESS in real time, a reinforcement learning algorithm featuring an experience replay mechanism is developed. Utilizing pretrained agents that integrate vehicle driving state abstraction parameters, the system generates the reference SOC of UC, enabling the optimal battery-UC power split in real time. Both numerical and semiphysical validations confirm the efficacy of the proposed strategy in enhancing the power output ratio of UC, optimizing energy storage space utilization, and reducing the battery severity factor, consequently improving overall battery lifespan.
AbstractList	The integration of ultracapacitors (UCs) into hybrid energy storage systems is a solution to mitigate battery degradation. Traditional strategies focus on fuel cell and battery power regulation while treating UC management as a passive element, resulting in suboptimal UC utilization. To optimize the energy utilization of UCs, this article proposes an active state control strategy within the hybrid system. Initially, leveraging the battery severity factor, the optimal power split strategy for HESS is proposed for a reference state-of-charge (SOC) of UC. Subsequently, a driving pattern severity factor is designed, and an online self-learning Markov predictor is employed to quantify the operational state of vehicle. To provide optimal reference SOC guidance to HESS in real time, a reinforcement learning algorithm featuring an experience replay mechanism is developed. Utilizing pretrained agents that integrate vehicle driving state abstraction parameters, the system generates the reference SOC of UC, enabling the optimal battery-UC power split in real time. Both numerical and semiphysical validations confirm the efficacy of the proposed strategy in enhancing the power output ratio of UC, optimizing energy storage space utilization, and reducing the battery severity factor, consequently improving overall battery lifespan.
Author	Xu, Xinhao Huang, Hao Lin, Xinyou
Author_xml	– sequence: 1 givenname: Xinyou orcidid: 0000-0002-4061-1055 surname: Lin fullname: Lin, Xinyou email: linxinyoou@fzu.edu.cn organization: School of Mechanical Engineering and Auto Mobile, Fuzhou University, Fuzhou, China – sequence: 2 givenname: Hao orcidid: 0009-0007-7228-4208 surname: Huang fullname: Huang, Hao email: 230220018@fzu.edu.cn organization: School of Mechanical Engineering and Auto Mobile, Fuzhou University, Fuzhou, China – sequence: 3 givenname: Xinhao orcidid: 0009-0002-4107-8026 surname: Xu fullname: Xu, Xinhao email: xinhaoxu@tesla.com organization: School of Mechanical Engineering and Auto Mobile, Fuzhou University, Fuzhou, China
BookMark	eNpNkDFvwjAQRq2KSgXavUMHS51DbcdJnJFSWpCQkArMkZNcUiNi04uplK0_vUEwdLrhvved7o3IwDoLhDxyNuGcpS_b5XwimJCTUCquFLshQx5FSZCmUg3IkIlEBYzJ-I6M2nbPGJcRj4bk91V7D9jRN6hRl9obZ-kaDVgPJZ0W3vwAnTnr0R3oxqP2UHc07-iuNbammn6CsZXDApoeoSvQaM-L6aF2aPxXQ42liy5HU9K5BezhjXeoa6CbrvXQ3JPbSh9aeLjOMdm9z7ezRbBafyxn01VQCBn5QLEqjHSpmI6SuEi4znmSJ6LgoArVfyzjKg5LUYo01zHPZSpFXGiuq7LKdSohHJPnS-8R3fcJWp_t3QltfzILe19c8FCKPsUuqQJd2yJU2RFNo7HLOMvOnrPec3b2nF0998jTBTEA8C-eSBaqNPwD1e181w
CODEN	ITIED6
Cites_doi	10.1016/j.ijepes.2012.06.026 10.1109/TIE.2015.2475419 10.1016/j.epsr.2023.109235 10.1109/TITS.2022.3233564 10.1016/j.apenergy.2019.113707 10.1109/TMECH.2018.2832019 10.1109/TIE.2020.2977574 10.1016/j.jpowsour.2019.04.083 10.1109/TTE.2015.2471180 10.1109/TPEL.2020.3028154 10.1109/TMECH.2017.2707338 10.1016/j.enconman.2022.116383 10.1016/j.energy.2023.129128 10.1016/j.jpowsour.2020.227964 10.1109/TIE.2020.2979528 10.1109/TVT.2022.3168870 10.1016/j.ijhydene.2020.01.017 10.1016/j.energy.2024.130728 10.1016/j.jpowsour.2018.05.041 10.1109/TVT.2020.3028089 10.1109/TIE.2021.3062273 10.1109/TIE.2018.2850030 10.1016/j.energy.2022.125620 10.1016/j.energy.2023.129173 10.1016/j.energy.2021.121593
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025
DBID	97E RIA RIE AAYXX CITATION 7SP 8FD L7M
DOI	10.1109/TIE.2024.3481880
DatabaseName	IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE/IET Electronic Library (IEL) CrossRef Electronics & Communications Abstracts Technology Research Database Advanced Technologies Database with Aerospace
DatabaseTitle	CrossRef Technology Research Database Advanced Technologies Database with Aerospace Electronics & Communications Abstracts
DatabaseTitleList	Technology Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	1557-9948
EndPage	4932
ExternalDocumentID	10_1109_TIE_2024_3481880 10740389
Genre	orig-research
GrantInformation_xml	– fundername: National Natural Science Foundation of China grantid: 52272389
GroupedDBID	-~X .DC 0R~ 29I 4.4 5GY 5VS 6IK 97E 9M8 AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK ACKIV ACNCT AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD HZ~ H~9 IBMZZ ICLAB IFIPE IFJZH IPLJI JAVBF LAI M43 MS~ O9- OCL P2P RIA RIE RNS TAE TN5 TWZ VH1 VJK AAYXX CITATION 7SP 8FD L7M
ID	FETCH-LOGICAL-c245t-80f35ad80a576c71ab17b72c1e8c848146f63d2d29ba61b49426ca1afdfba94e3
IEDL.DBID	RIE
ISICitedReferencesCount	1
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001346681900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0278-0046
IngestDate	Tue Jul 22 16:40:56 EDT 2025 Sat Nov 29 08:04:56 EST 2025 Wed Aug 27 02:04:40 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	5
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c245t-80f35ad80a576c71ab17b72c1e8c848146f63d2d29ba61b49426ca1afdfba94e3
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0009-0002-4107-8026 0009-0007-7228-4208 0000-0002-4061-1055
PQID	3188121342
PQPubID	85464
PageCount	11
ParticipantIDs	proquest_journals_3188121342 ieee_primary_10740389 crossref_primary_10_1109_TIE_2024_3481880
PublicationCentury	2000
PublicationDate	2025-05-01
PublicationDateYYYYMMDD	2025-05-01
PublicationDate_xml	– month: 05 year: 2025 text: 2025-05-01 day: 01
PublicationDecade	2020
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	IEEE transactions on industrial electronics (1982)
PublicationTitleAbbrev	TIE
PublicationYear	2025
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 ref12 ref15 ref14 ref11 ref10 ref2 Sheng (ref6) 2022; 47 ref1 ref17 ref16 ref19 ref18 ref24 ref23 ref26 ref25 ref20 ref22 ref21 ref8 ref7 ref9 ref4 ref3 ref5
References_xml	– ident: ref10 doi: 10.1016/j.ijepes.2012.06.026 – ident: ref17 doi: 10.1109/TIE.2015.2475419 – ident: ref12 doi: 10.1016/j.epsr.2023.109235 – ident: ref23 doi: 10.1109/TITS.2022.3233564 – ident: ref24 doi: 10.1016/j.apenergy.2019.113707 – ident: ref9 doi: 10.1109/TMECH.2018.2832019 – ident: ref1 doi: 10.1109/TIE.2020.2977574 – ident: ref7 doi: 10.1016/j.jpowsour.2019.04.083 – ident: ref25 doi: 10.1109/TTE.2015.2471180 – ident: ref4 doi: 10.1109/TPEL.2020.3028154 – ident: ref18 doi: 10.1109/TMECH.2017.2707338 – ident: ref15 doi: 10.1016/j.enconman.2022.116383 – ident: ref16 doi: 10.1016/j.energy.2023.129128 – ident: ref20 doi: 10.1016/j.jpowsour.2020.227964 – ident: ref2 doi: 10.1109/TIE.2020.2979528 – ident: ref21 doi: 10.1109/TVT.2022.3168870 – ident: ref5 doi: 10.1016/j.ijhydene.2020.01.017 – ident: ref26 doi: 10.1016/j.energy.2024.130728 – volume: 47 year: 2022 ident: ref6 article-title: Experimental investigation of supercapacitor based regenerative energy storage for a fuel cell vehicle equipped with an alternator publication-title: Int. J. Hydrogen Energy – ident: ref13 doi: 10.1016/j.jpowsour.2018.05.041 – ident: ref14 doi: 10.1109/TVT.2020.3028089 – ident: ref22 doi: 10.1109/TIE.2021.3062273 – ident: ref3 doi: 10.1109/TIE.2018.2850030 – ident: ref8 doi: 10.1016/j.energy.2022.125620 – ident: ref11 doi: 10.1016/j.energy.2023.129173 – ident: ref19 doi: 10.1016/j.energy.2021.121593
SSID	ssj0014515
Score	2.4812686
Snippet	The integration of ultracapacitors (UCs) into hybrid energy storage systems is a solution to mitigate battery degradation. Traditional strategies focus on fuel...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Index Database Publisher
StartPage	4922
SubjectTerms	Active control Algorithms Batteries Control systems Deep reinforcement learning (RL) Degradation Electric charge Energy management Energy storage Energy utilization fuel cell hybrid electric vehicle (FCHEV) Fuel cells hybrid energy storage system (HESS) Hybrid systems Machine learning Numerical models Optimization Power generation Real time Real-time systems State of charge
Title	Battery Degradation Oriented Active Control Strategy by Using a Reinforcement Learning Algorithm in Hybrid Energy Storage System
URI	https://ieeexplore.ieee.org/document/10740389 https://www.proquest.com/docview/3188121342
Volume	72
WOSCitedRecordID	wos001346681900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1557-9948 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014515 issn: 0278-0046 databaseCode: RIE dateStart: 19820101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlZ07T8MwEMctWjHAwLOIQkE3sDCkrR0nTsaqFJWlIFSkbpHt2KUSpKgPpGx8dPxIUSXEwJYhiSLfxeez7_87hG6wMp5gjBtEIuQBZVoFPIlxIBQnIYsw0yR3zSbYaJRMJulTJVZ3WhillCs-U2176c7y87lc262yji0etEC4GqoxFnux1s-RAY18uwJikbEm69ucSXbTzvhhYDJBQttWdZpYAuRWDHJNVX7NxC683B_-88OO0EG1joSeN_wx2lHFCdrfogueoi_PzizhzgIhfO8keLRcY7PKhJ6b6KDvS9WhotSWIEpwZQTA4Vk5rKp0O4hQkVin0Hubzhez1es7zAoYllbyBQMnITRvMR41VeA56A30cj8Y94dB1XAhkIRGKxOtdBjxPOlyk4VIhrnATDAisUqkxe7TWMdhTnKSCh5jQVMT3iXHXOda8JSq8AzVi3mhzhEkOSV5HBMuKaOSqFRbvYOVa5oljxa6iW43Jsg-PFcjc_lIN82MuTJrrqwyVxM17JBv3edHu4laG6Nl1Z-3zMwclThMHbn447FLtEdsE19XtdhC9dVira7QrvxczZaLa-dU38ovy2I
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlZ07T8MwEMdPUJCAgTeiPG9gYQg0jvMaq1LUCigIFYktsh27VIIWlRQpGx8dP1JUCTGwZUicyOf4fPb9fwdw5ks9ErRxvZAHzKOxkh5LIt_jkpEgDv1YkdwWm4h7veT5OX2oxOpWCyOltMln8sJc2rP8fCymZqvs0iQPGiDcIiyFVL_CybV-Dg1o6AoWEAON1XHf7FSykV72u20dCxJ6YXSniWFAznkhW1bl11xsHcz1xj8_bRPWq5UkNp3pt2BBjrZhbY4vuANfjp5Z4pVBQrjqSXhvyMZ6nYlNO9VhyyWrY8WpLZGXaBMJkOGjtGBVYfcQsWKxDrD5OhhPhsXLGw5H2CmN6AvbVkSoW9FjaiDRkdB34em63W91vKrkgicIDQvtr1QQsjxpMB2HiNhn3I95TIQvE2HA-zRSUZCTnKScRT6nqXbwgvlM5YqzlMpgD2qj8UjuAyY5JXkUESZoTAWRqTKKByPY1IsexVUdzmcmyN4dWSOzEUkjzbS5MmOurDJXHXZNl8_d53q7Dkczo2XVv_eR6VkqsaA6cvDHY6ew0unf3Wa33d7NIawSU9LX5jAeQa2YTOUxLIvPYvgxObED7BuGWc6p
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Battery+Degradation+Oriented+Active+Control+Strategy+by+Using+a+Reinforcement+Learning+Algorithm+in+Hybrid+Energy+Storage+System&rft.jtitle=IEEE+transactions+on+industrial+electronics+%281982%29&rft.au=Lin%2C+Xinyou&rft.au=Huang%2C+Hao&rft.au=Xu%2C+Xinhao&rft.date=2025-05-01&rft.pub=IEEE&rft.issn=0278-0046&rft.volume=72&rft.issue=5&rft.spage=4922&rft.epage=4932&rft_id=info:doi/10.1109%2FTIE.2024.3481880&rft.externalDocID=10740389
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0278-0046&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0278-0046&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0278-0046&client=summon