An Integrated Decision-Making Framework for Highway Autonomous Driving Using Combined Learning and Rule-Based Algorithm

In order to solve the manual labelling, long-tail effect and driving conservatism of the existing decision-making algorithm. This paper proposed an integrated decision-making framework (IDF) for highway autonomous vehicles. Firstly, states of the highway traffic are extracted by the velocity, time h...

Full description

Saved in:

Bibliographic Details
Published in:	IEEE transactions on vehicular technology Vol. 71; no. 4; pp. 3621 - 3632
Main Authors:	Xu, Can, Zhao, Wanzhong, Liu, Jinqiang, Wang, Chunyan, Lv, Chen
Format:	Journal Article
Language:	English
Published:	New York IEEE 01.04.2022 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:	Algorithms Artificial neural networks Autonomous vehicles Decision making Driving Headways highway driving Machine learning Roads Task analysis Training Trajectory Windows
ISSN:	0018-9545, 1939-9359
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Abstract	In order to solve the manual labelling, long-tail effect and driving conservatism of the existing decision-making algorithm. This paper proposed an integrated decision-making framework (IDF) for highway autonomous vehicles. Firstly, states of the highway traffic are extracted by the velocity, time headway (TH) and the probabilistic lane distribution of the surrounding vehicles. With the extracted traffic state, the reinforcement learning (RL) is adopted to learn the optimal state-action pair for specific scenario. Analogously, by mapping millions of traffic scenarios, huge amounts of state-action pairs can be stored in the experience pool. Then the imitation learning (IL) is further employed to memorize the experience pool by deep neural networks. The learning result shows that the accuracy of the decision network can reach 94.17%. Besides, for some imperfect decisions of the network, the rule-based method is taken to rectify by judging the long-term reward. Finally, the IDF is simulated in G25 highway and has promising results, which can always drive the vehicle to the state with high efficiency while ensuring safety.
AbstractList	In order to solve the manual labelling, long-tail effect and driving conservatism of the existing decision-making algorithm. This paper proposed an integrated decision-making framework (IDF) for highway autonomous vehicles. Firstly, states of the highway traffic are extracted by the velocity, time headway (TH) and the probabilistic lane distribution of the surrounding vehicles. With the extracted traffic state, the reinforcement learning (RL) is adopted to learn the optimal state-action pair for specific scenario. Analogously, by mapping millions of traffic scenarios, huge amounts of state-action pairs can be stored in the experience pool. Then the imitation learning (IL) is further employed to memorize the experience pool by deep neural networks. The learning result shows that the accuracy of the decision network can reach 94.17%. Besides, for some imperfect decisions of the network, the rule-based method is taken to rectify by judging the long-term reward. Finally, the IDF is simulated in G25 highway and has promising results, which can always drive the vehicle to the state with high efficiency while ensuring safety.
Author	Xu, Can Lv, Chen Liu, Jinqiang Zhao, Wanzhong Wang, Chunyan
Author_xml	– sequence: 1 givenname: Can orcidid: 0000-0001-7551-3394 surname: Xu fullname: Xu, Can email: xucan2021@163.com organization: Department of Vehicle Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China – sequence: 2 givenname: Wanzhong orcidid: 0000-0002-8750-3553 surname: Zhao fullname: Zhao, Wanzhong email: zhaowanzhong@126.com organization: Department of Vehicle Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China – sequence: 3 givenname: Jinqiang orcidid: 0000-0002-9472-627X surname: Liu fullname: Liu, Jinqiang email: jinqiang916@163.com organization: Department of Vehicle Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China – sequence: 4 givenname: Chunyan surname: Wang fullname: Wang, Chunyan email: wcy2000@126.com organization: Department of Vehicle Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China – sequence: 5 givenname: Chen orcidid: 0000-0001-6897-4512 surname: Lv fullname: Lv, Chen email: lyuchen@ntu.edu.sg organization: School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore
BookMark	eNp9kM1Lw0AQxRdRsFXvgpeA59T9SJrssbZWhYog1WvYbCZx22RXdzeW_vduqHjw4GWGGd6bx_zG6FgbDQhdEjwhBPOb9dt6QjGlE0ZSzBJ2hEaEMx5zlvJjNMKY5DFPk_QUjZ3bhDFJOBmh3UxHj9pDY4WHKlqAVE4ZHT-JrdJNtLSig52x26g2NnpQzftO7KNZ7402neldtLDqaxC-uqHOTVcqHe6sQFg9bISuope-hfhWuLCftY2xyr935-ikFq2Di59-hl6Xd-v5Q7x6vn-cz1axpJz4OGM8LzMKWGR5WkqZ8LwqMaZS5CXFXIIknMo6KWnOCa1SBgwqKphgMpF1mM7Q9eHuhzWfPThfbExvdYgs6DTlLMkongYVPqikNc5ZqIsPqzph9wXBxYC3CHiLAW_xgzdYpn8sUnnhAztvhWr_M14djAoAfnN4Rmh4iH0DR92KrA
CODEN	ITVTAB
CitedBy_id	crossref_primary_10_1109_ACCESS_2025_3570609 crossref_primary_10_1109_TVT_2025_3554978 crossref_primary_10_1016_j_isatra_2025_01_033 crossref_primary_10_1109_TVT_2023_3311198 crossref_primary_10_3390_electronics13204006 crossref_primary_10_1007_s10489_025_06319_2 crossref_primary_10_1109_TVT_2024_3377288 crossref_primary_10_1016_j_robot_2025_105180 crossref_primary_10_1109_TVT_2024_3398661 crossref_primary_10_1007_s40430_023_04458_6 crossref_primary_10_3390_act14070315 crossref_primary_10_1016_j_tra_2024_104069 crossref_primary_10_1109_TITS_2022_3227122 crossref_primary_10_1049_itr2_12507 crossref_primary_10_1109_ACCESS_2024_3406260 crossref_primary_10_1109_TVT_2023_3285223 crossref_primary_10_3390_su16114578 crossref_primary_10_1109_TASE_2025_3586434 crossref_primary_10_1016_j_conengprac_2025_106315 crossref_primary_10_1109_TMECH_2023_3313170 crossref_primary_10_3390_wevj15110489 crossref_primary_10_1016_j_geits_2025_100288 crossref_primary_10_1109_TIV_2024_3430484 crossref_primary_10_1109_TVT_2023_3298635 crossref_primary_10_1007_s11082_023_05752_2 crossref_primary_10_1007_s12239_025_00351_4 crossref_primary_10_1016_j_trc_2024_104654 crossref_primary_10_1080_23307706_2025_2521782 crossref_primary_10_1177_09544070231187687
Cites_doi	10.1109/TMECH.2015.2493980 10.1109/TVT.2019.2948953 10.1109/ITSC.2018.8569568 10.1109/ICRA.2019.8793698 10.1109/ICCV.2019.00942 10.1109/TITS.2011.2157145 10.1109/TVT.2016.2555853 10.1109/TVT.2020.3027352 10.1109/JAS.2018.7511186 10.1109/TITS.2017.2768318 10.1007/978-3-030-01225-0_33 10.1109/TVT.2018.2820002 10.1109/TVT.2019.2945934 10.1049/iet-its.2019.0317 10.1016/j.conengprac.2012.09.020 10.1109/TITS.2019.2913998 10.1016/j.trc.2018.10.024 10.1109/JAS.2019.1911825 10.1109/TVT.2018.2822762 10.1109/TITS.2019.2919865 10.1109/TITS.2019.2918117 10.1109/TVT.2019.2930684 10.1115/DSCC2015-9773 10.1109/TITS.2006.883115 10.1109/JAS.2020.1003021
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022
DBID	97E RIA RIE AAYXX CITATION 7SP 8FD FR3 KR7 L7M
DOI	10.1109/TVT.2022.3150343
DatabaseName	IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Electronics & Communications Abstracts Technology Research Database Engineering Research Database Civil Engineering Abstracts Advanced Technologies Database with Aerospace
DatabaseTitle	CrossRef Civil Engineering Abstracts Engineering Research Database Technology Research Database Advanced Technologies Database with Aerospace Electronics & Communications Abstracts
DatabaseTitleList	Civil Engineering Abstracts
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	1939-9359
EndPage	3632
ExternalDocumentID	10_1109_TVT_2022_3150343 9712209
Genre	orig-research
GrantInformation_xml	– fundername: National Natural Science Foundation of China grantid: 52072175; 51775007 funderid: 10.13039/501100001809 – fundername: China Scholarship Council grantid: 202006830050 funderid: 10.13039/501100004543
GroupedDBID	-~X .DC 0R~ 29I 3EH 4.4 5GY 5VS 6IK 97E AAIKC AAJGR AAMNW AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK ACNCT AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD HZ~ H~9 IAAWW IBMZZ ICLAB IFIPE IFJZH IPLJI JAVBF LAI M43 MS~ O9- OCL P2P RIA RIE RNS RXW TAE TN5 VH1 AAYXX CITATION 7SP 8FD FR3 KR7 L7M
ID	FETCH-LOGICAL-c291t-7398b72e0a785bcc498db002ca8b209cec192cf4b28912d53e3ed2a3a3c4cf3e3
IEDL.DBID	RIE
ISICitedReferencesCount	39
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000790830700022&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0018-9545
IngestDate	Mon Jun 30 10:07:46 EDT 2025 Sat Nov 29 02:59:00 EST 2025 Tue Nov 18 20:38:48 EST 2025 Wed Aug 27 02:40:12 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	4
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c291t-7398b72e0a785bcc498db002ca8b209cec192cf4b28912d53e3ed2a3a3c4cf3e3
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0002-8750-3553 0000-0002-9472-627X 0000-0001-7551-3394 0000-0001-6897-4512
PQID	2659347206
PQPubID	85454
PageCount	12
ParticipantIDs	ieee_primary_9712209 proquest_journals_2659347206 crossref_citationtrail_10_1109_TVT_2022_3150343 crossref_primary_10_1109_TVT_2022_3150343
PublicationCentury	2000
PublicationDate	2022-04-01
PublicationDateYYYYMMDD	2022-04-01
PublicationDate_xml	– month: 04 year: 2022 text: 2022-04-01 day: 01
PublicationDecade	2020
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	IEEE transactions on vehicular technology
PublicationTitleAbbrev	TVT
PublicationYear	2022
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 ref12 ref15 ref14 ref30 ref11 ref10 ref2 ref1 Bojarski (ref23) 2016 ref17 ref19 ref18 Ros (ref16) Bojarski (ref22) 2017 (ref24) 2004 ref26 ref25 Schulman (ref31) 2017 ref21 Rehder (ref20) 2017 ref28 ref27 ref29 ref8 ref7 ref9 ref4 ref3 ref6 ref5
References_xml	– ident: ref3 doi: 10.1109/TMECH.2015.2493980 – year: 2017 ident: ref31 article-title: Proximal policy optimization algorithms – year: 2016 ident: ref23 article-title: End to end learning for self-driving cars – start-page: 1 volume-title: Proc. IEEE Int. Conf. Intell. Robots Syst. year: 2017 ident: ref20 article-title: Driving like a human: Imitation learning for path planning using convolutional neural networks – ident: ref11 doi: 10.1109/TVT.2019.2948953 – ident: ref15 doi: 10.1109/ITSC.2018.8569568 – ident: ref19 doi: 10.1109/ICRA.2019.8793698 – year: 2017 ident: ref22 article-title: Explaining how a deep neural network trained with end-to-end learning steers a car – ident: ref27 doi: 10.1109/ICCV.2019.00942 – ident: ref13 doi: 10.1109/TITS.2011.2157145 – ident: ref2 doi: 10.1109/TVT.2016.2555853 – ident: ref29 doi: 10.1109/TVT.2020.3027352 – ident: ref18 doi: 10.1109/JAS.2018.7511186 – ident: ref28 doi: 10.1109/TITS.2017.2768318 – ident: ref25 doi: 10.1007/978-3-030-01225-0_33 – ident: ref14 doi: 10.1109/TVT.2018.2820002 – start-page: 1 volume-title: Proc. 14th Int. Conf. Artif. Intell. Statist. ident: ref16 article-title: A reduction of imitation learning and structured prediction to no-regret online learning – ident: ref5 doi: 10.1109/TVT.2019.2945934 – ident: ref9 doi: 10.1049/iet-its.2019.0317 – ident: ref8 doi: 10.1016/j.conengprac.2012.09.020 – ident: ref1 doi: 10.1109/TITS.2019.2913998 – ident: ref12 doi: 10.1016/j.trc.2018.10.024 – ident: ref17 doi: 10.1109/JAS.2019.1911825 – ident: ref10 doi: 10.1109/TVT.2018.2822762 – ident: ref30 doi: 10.1109/TITS.2019.2919865 – year: 2004 ident: ref24 article-title: Autonomous off-road vehicle control using end-to-end learning – ident: ref26 doi: 10.1109/TITS.2019.2918117 – ident: ref4 doi: 10.1109/TVT.2019.2930684 – ident: ref7 doi: 10.1115/DSCC2015-9773 – ident: ref6 doi: 10.1109/TITS.2006.883115 – ident: ref21 doi: 10.1109/JAS.2020.1003021
SSID	ssj0014491
Score	2.5286136
Snippet	In order to solve the manual labelling, long-tail effect and driving conservatism of the existing decision-making algorithm. This paper proposed an integrated...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	3621
SubjectTerms	Algorithms Artificial neural networks Autonomous vehicles Decision making Driving Headways highway driving Machine learning Roads Task analysis Training Trajectory Windows
Title	An Integrated Decision-Making Framework for Highway Autonomous Driving Using Combined Learning and Rule-Based Algorithm
URI	https://ieeexplore.ieee.org/document/9712209 https://www.proquest.com/docview/2659347206
Volume	71
WOSCitedRecordID	wos000790830700022&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1939-9359 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014491 issn: 0018-9545 databaseCode: RIE dateStart: 19670101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NT9swGLYY4rAdxoBN62CTD1yQMHUcN7aPZVCNAwihMnGL7NcOVCrp1Car9u-xXadiYprELZFsK8kTvx9-Px6EDq2lxhsOmlBnKOFFoYgG6UjFtcg4OC84IZJNiKsreXenrjfQ8boWxjkXk8_cSbiMsXw7gzYclfWVyBgL1XpvhChWtVrriAHniR0v8xvYmwVdSJKq_vjn2DuCjHn_dEBznv-lgiKnygtBHLXLaPt1z_UBvU9WJB6uYN9BG67eRe-e9RbcQ8thjS-6XhAWnyUuHXIZ6afwqEvKwt5qxSHbY6n_4GHbhCKHWbvAZ_NJOGvAMacAe7HhXWi_TurHeo91bfFNO3Xk1OtBi4fT-9l80jw8fkS3o_Px9x8ksSwQYCpriMiVNII5qoUcGACupA2qG7Q0_qXAgccSKm68a5YxO8hd7izTuc6BQ-XvPqHNela7zwgLzRTlRlGjHa9MJSsqoShoRS1IZXkP9bsPX0JqQR6YMKZldEWoKj1UZYCqTFD10NF6xq9V-43_jN0L0KzHJVR66KDDtkz7c1GyYqByLhgtvvx71j56G9Ze5egcoM1m3rqvaAt-N5PF_Fv89Z4AdDXYHA
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1bb9MwFD6aNiTYw7gMRGGAH3hBwtRx3MR-LIxqE1uFUEF7i-xjZ1Qq6dQmTPv3s12nAoGQeEskO7cvPhefywfw2lpmvOGgKXOGUVEUimqUjtZCl5lA5wUnRrKJcjqVFxfq8w683dbCOOdi8pl7Fw5jLN8usQtbZUNVZpyHar29wJyVqrW2MQMhEj9e5pewNwz6oCRTw9m3mXcFOfce6ojlIv9NCUVWlT9EcdQvk_v_92QP4CDZkWS8Af4h7LjmEez_0l3wEK7HDTntu0FYcpzYdOh5JKAikz4ti3i7lYR8j2t9Q8ZdG8oclt2aHK_mYbeBxKwC4gWHd6L9dVJH1kuiG0u-dAtH33tNaMl4cblczdvvPx7D18nH2YcTmngWKHKVtbTMlTQld0yXcmQQhZI2KG_U0viXQoceTayF8c5Zxu0od7mzXOc6R4G1P3sCu82ycU-BlJorJoxiRjtRm1rWTGJRsJpZlMqKAQz7D19hakIeuDAWVXRGmKo8VFWAqkpQDeDNdsbVpgHHP8YeBmi24xIqAzjqsa3SCl1XvBipXJScFc_-PusV3D2ZnZ9VZ6fTT8_hXrjPJmPnCHbbVedewB382c7Xq5fxN7wF3OrbZQ
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=An+Integrated+Decision-Making+Framework+for+Highway+Autonomous+Driving+Using+Combined+Learning+and+Rule-Based+Algorithm&rft.jtitle=IEEE+transactions+on+vehicular+technology&rft.au=Xu%2C+Can&rft.au=Zhao%2C+Wanzhong&rft.au=Liu%2C+Jinqiang&rft.au=Wang%2C+Chunyan&rft.date=2022-04-01&rft.pub=IEEE&rft.issn=0018-9545&rft.volume=71&rft.issue=4&rft.spage=3621&rft.epage=3632&rft_id=info:doi/10.1109%2FTVT.2022.3150343&rft.externalDocID=9712209
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9545&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9545&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9545&client=summon