An optimized Q-Learning algorithm for mobile robot local path planning
The Q-Learning algorithm is a reinforcement learning technique widely used in various fields such as path planning, intelligent transportation, penetration testing, among others. It primarily involves the interaction between an agent and its environment, enabling the agent to learn an optimal strate...
Gespeichert in:
| Veröffentlicht in: | Knowledge-based systems Jg. 286; S. 111400 |
|---|---|
| Hauptverfasser: | , , , , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
Elsevier B.V
28.02.2024
|
| Schlagworte: | |
| ISSN: | 0950-7051, 1872-7409 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | The Q-Learning algorithm is a reinforcement learning technique widely used in various fields such as path planning, intelligent transportation, penetration testing, among others. It primarily involves the interaction between an agent and its environment, enabling the agent to learn an optimal strategy that maximizes cumulative rewards. Most non-agent-based path planning algorithms face challenges in exploring completely unknown environments effectively, lacking efficient perception in unfamiliar settings. Additionally, many Q-Learning-based path planning algorithms suffer from slow convergence and susceptibility to getting stuck in local optimal solutions. To address these issues, an optimized version of the Q-Learning algorithm (Optimized Q-Learning, O-QL) is proposed and applied to local path planning of mobile robots. O-QL introduces novel Q-table initialization methods, incorporates a new action-selection policy, and a new reward function, and adapts the Root Mean Square Propagation (RMSprop) method in the learning rate adjustment. This adjustment dynamically tunes the learning rate based on gradient changes to accelerate learning and enhance path planning efficiency. Simulation experiments are carried out in three maze environments with different complexity levels, and the performance of the algorithm in local path planning is evaluated using steps, exploration reward, learning rate change and running time. The experimental results demonstrate that O-QL exhibits improvements across all four metrics compared to existing algorithms. |
|---|---|
| AbstractList | The Q-Learning algorithm is a reinforcement learning technique widely used in various fields such as path planning, intelligent transportation, penetration testing, among others. It primarily involves the interaction between an agent and its environment, enabling the agent to learn an optimal strategy that maximizes cumulative rewards. Most non-agent-based path planning algorithms face challenges in exploring completely unknown environments effectively, lacking efficient perception in unfamiliar settings. Additionally, many Q-Learning-based path planning algorithms suffer from slow convergence and susceptibility to getting stuck in local optimal solutions. To address these issues, an optimized version of the Q-Learning algorithm (Optimized Q-Learning, O-QL) is proposed and applied to local path planning of mobile robots. O-QL introduces novel Q-table initialization methods, incorporates a new action-selection policy, and a new reward function, and adapts the Root Mean Square Propagation (RMSprop) method in the learning rate adjustment. This adjustment dynamically tunes the learning rate based on gradient changes to accelerate learning and enhance path planning efficiency. Simulation experiments are carried out in three maze environments with different complexity levels, and the performance of the algorithm in local path planning is evaluated using steps, exploration reward, learning rate change and running time. The experimental results demonstrate that O-QL exhibits improvements across all four metrics compared to existing algorithms. |
| ArticleNumber | 111400 |
| Author | Wu, Jiayang Zhu, Mengyue Zhou, Qian Cao, Jinli Lian, Yang Wang, Haiyong |
| Author_xml | – sequence: 1 givenname: Qian orcidid: 0000-0001-7888-2419 surname: Zhou fullname: Zhou, Qian email: zhouqian@njupt.edu.cn organization: School of Modern Posts, Nanjing University of Posts and Telecommunications, Nanjing, 210003, Jiangsu, China – sequence: 2 givenname: Yang surname: Lian fullname: Lian, Yang email: 1321048425@njupt.edu.cn organization: School of Computer Science, School of Software and School of Cyberspace Security, Nanjing University of Posts and Telecommunications, Nanjing, 210037, Jiangsu, China – sequence: 3 givenname: Jiayang surname: Wu fullname: Wu, Jiayang email: 1223096926@njupt.edu.cn organization: School of Modern Posts, Nanjing University of Posts and Telecommunications, Nanjing, 210003, Jiangsu, China – sequence: 4 givenname: Mengyue surname: Zhu fullname: Zhu, Mengyue email: 1223096927@njupt.edu.cn organization: School of Modern Posts, Nanjing University of Posts and Telecommunications, Nanjing, 210003, Jiangsu, China – sequence: 5 givenname: Haiyong surname: Wang fullname: Wang, Haiyong email: why@njupt.edu.cn organization: School of Computer Science, School of Software and School of Cyberspace Security, Nanjing University of Posts and Telecommunications, Nanjing, 210037, Jiangsu, China – sequence: 6 givenname: Jinli orcidid: 0000-0002-0221-6361 surname: Cao fullname: Cao, Jinli email: J.Cao@latrobe.edu.au organization: Department of Computer Science and Computer Engineering, La Trobe University, Melbourne, Australia |
| BookMark | eNqFkM1KAzEUhYNUsK2-gYu8wIw3aebPhVCKrUJBBF2HTH7a1JlkSIJQn96WceVCV5cD9ztwvhmaOO80QrcEcgKkvDvkH87HY8wpUJYTQhjABZqSuqJZxaCZoCk0BWQVFOQKzWI8AAClpJ6i9dJhPyTb2y-t8Gu21SI463ZYdDsfbNr32PiAe9_aTuPgW59w56Xo8CDSHg-dcOf3a3RpRBf1zc-do_f149vqKdu-bJ5Xy20mF1CmTIGpC1a2LasFCKaUqEnZCKYFhaZitJWFEaquTN0Qs1CMsgaYPuVWlopps5ij-7FXBh9j0IZLm0Sy3qUgbMcJ8LMRfuCjEX42wkcjJ5j9godgexGO_2EPI6ZPwz6tDjxKq53UygYtE1fe_l3wDUppgA0 |
| CitedBy_id | crossref_primary_10_1002_rob_22531 crossref_primary_10_3390_app15031165 crossref_primary_10_1016_j_ijhydene_2024_11_262 crossref_primary_10_1016_j_knosys_2024_112120 crossref_primary_10_3934_jimo_2025150 crossref_primary_10_1016_j_compeleceng_2024_110034 crossref_primary_10_1109_JSEN_2025_3557437 crossref_primary_10_1016_j_neucom_2024_128423 crossref_primary_10_1016_j_knosys_2024_112326 crossref_primary_10_1109_ACCESS_2025_3607976 crossref_primary_10_1109_ACCESS_2025_3602286 crossref_primary_10_1016_j_eswa_2024_125388 crossref_primary_10_1088_1402_4896_adb79a crossref_primary_10_3390_electronics14163199 crossref_primary_10_1016_j_knosys_2024_112406 crossref_primary_10_1109_TASE_2025_3604604 crossref_primary_10_1007_s11227_025_07318_3 crossref_primary_10_1016_j_engappai_2024_109866 crossref_primary_10_1016_j_knosys_2025_113836 crossref_primary_10_1016_j_ijtst_2025_06_001 crossref_primary_10_1017_S0263574725000529 crossref_primary_10_1016_j_chaos_2025_116656 crossref_primary_10_1016_j_ast_2025_110950 crossref_primary_10_1016_j_physa_2025_130902 crossref_primary_10_1016_j_neucom_2025_131173 crossref_primary_10_1002_rob_22479 crossref_primary_10_1016_j_aei_2025_103398 crossref_primary_10_48084_etasr_11986 crossref_primary_10_1109_TKDE_2025_3579386 crossref_primary_10_1016_j_ress_2025_111030 crossref_primary_10_1007_s11227_025_07470_w crossref_primary_10_1080_14484846_2025_2514394 crossref_primary_10_1016_j_engappai_2025_111890 crossref_primary_10_3390_drones8090462 crossref_primary_10_1016_j_rineng_2025_105325 crossref_primary_10_1109_ACCESS_2025_3602149 crossref_primary_10_1109_TAFE_2025_3528403 crossref_primary_10_1177_09544062241281115 crossref_primary_10_1016_j_tbs_2024_100943 crossref_primary_10_3390_fishes10080420 crossref_primary_10_1016_j_jobe_2025_112626 crossref_primary_10_1088_2631_8695_adcb92 crossref_primary_10_1016_j_eswa_2025_127808 crossref_primary_10_1016_j_engappai_2025_111030 crossref_primary_10_2478_jaiscr_2025_0012 crossref_primary_10_1016_j_sftr_2025_100575 crossref_primary_10_1016_j_eswa_2025_128051 crossref_primary_10_1016_j_sasc_2025_200332 crossref_primary_10_1088_1361_6501_adee3a crossref_primary_10_1088_1402_4896_ad551b crossref_primary_10_32604_cmc_2024_056791 crossref_primary_10_1016_j_enganabound_2025_106407 crossref_primary_10_1109_JIOT_2024_3379361 crossref_primary_10_3390_technologies12060082 crossref_primary_10_1007_s11370_025_00617_x crossref_primary_10_3390_app142311195 |
| Cites_doi | 10.1016/j.knosys.2019.03.018 10.1109/ICASERT.2019.8934450 10.1016/j.knosys.2019.104938 10.1109/TSMCA.2012.2227719 10.1109/TITS.2019.2913998 10.1016/j.knosys.2023.110355 10.3390/sym10100450 10.1007/s10846-016-0362-z 10.1109/AIM.2009.5229903 10.1007/s10489-023-04946-1 10.1016/j.knosys.2022.109075 10.1016/j.eswa.2022.117191 10.1016/j.ymssp.2017.07.019 10.1109/CVCI56766.2022.9964859 10.1109/TCYB.2021.3107415 10.1023/A:1022633531479 10.1109/ICRAS.2018.8443226 10.1016/j.procs.2018.01.054 10.1007/978-3-642-16111-7_23 10.1007/s12652-018-0777-4 10.1023/A:1007694015589 10.1109/ICInfA.2015.7279322 10.1364/OE.494734 10.1109/TVT.2021.3093318 10.1109/TIE.2021.3062273 10.1016/j.robot.2019.02.013 10.1109/TVT.2014.2335201 |
| ContentType | Journal Article |
| Copyright | 2024 Elsevier B.V. |
| Copyright_xml | – notice: 2024 Elsevier B.V. |
| DBID | AAYXX CITATION |
| DOI | 10.1016/j.knosys.2024.111400 |
| DatabaseName | CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISSN | 1872-7409 |
| ExternalDocumentID | 10_1016_j_knosys_2024_111400 S0950705124000352 |
| GroupedDBID | --K --M .DC .~1 0R~ 1B1 1~. 1~5 4.4 457 4G. 5VS 7-5 71M 77K 8P~ 9JN AACTN AAEDT AAEDW AAIAV AAIKJ AAKOC AALRI AAOAW AAQFI AAXUO AAYFN ABAOU ABBOA ABIVO ABJNI ABMAC ABYKQ ACAZW ACDAQ ACGFS ACRLP ACZNC ADBBV ADEZE ADGUI ADTZH AEBSH AECPX AEKER AENEX AFKWA AFTJW AGHFR AGUBO AGYEJ AHHHB AHJVU AHZHX AIALX AIEXJ AIKHN AITUG AJOXV ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AOUOD ARUGR AXJTR BJAXD BKOJK BLXMC CS3 DU5 EBS EFJIC EFLBG EO8 EO9 EP2 EP3 FDB FIRID FNPLU FYGXN G-Q GBLVA GBOLZ IHE J1W JJJVA KOM MHUIS MO0 N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. PQQKQ Q38 ROL RPZ SDF SDG SDP SES SEW SPC SPCBC SST SSV SSW SSZ T5K WH7 XPP ZMT ~02 ~G- 29L 77I 9DU AAQXK AATTM AAXKI AAYWO AAYXX ABDPE ABWVN ABXDB ACLOT ACNNM ACRPL ACVFH ADCNI ADJOM ADMUD ADNMO AEIPS AEUPX AFJKZ AFPUW AGQPQ AIGII AIIUN AKBMS AKRWK AKYEP ANKPU APXCP ASPBG AVWKF AZFZN CITATION EFKBS EJD FEDTE FGOYB G-2 HLZ HVGLF HZ~ LG9 LY7 M41 R2- SBC SET UHS WUQ ~HD |
| ID | FETCH-LOGICAL-c306t-d0f8546bb48a0a4dda8169a4ea209742bc5fad87f891f3d424904e87fbc6d4ef3 |
| ISICitedReferencesCount | 68 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001170720800001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0950-7051 |
| IngestDate | Sat Nov 29 07:10:07 EST 2025 Tue Nov 18 22:24:52 EST 2025 Sat Feb 24 15:48:44 EST 2024 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Q-Learning algorithm Local path planning Adaptive learning rate Mobile robot Reinforcement learning |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c306t-d0f8546bb48a0a4dda8169a4ea209742bc5fad87f891f3d424904e87fbc6d4ef3 |
| ORCID | 0000-0001-7888-2419 0000-0002-0221-6361 |
| ParticipantIDs | crossref_citationtrail_10_1016_j_knosys_2024_111400 crossref_primary_10_1016_j_knosys_2024_111400 elsevier_sciencedirect_doi_10_1016_j_knosys_2024_111400 |
| PublicationCentury | 2000 |
| PublicationDate | 2024-02-28 |
| PublicationDateYYYYMMDD | 2024-02-28 |
| PublicationDate_xml | – month: 02 year: 2024 text: 2024-02-28 day: 28 |
| PublicationDecade | 2020 |
| PublicationTitle | Knowledge-based systems |
| PublicationYear | 2024 |
| Publisher | Elsevier B.V |
| Publisher_xml | – name: Elsevier B.V |
| References | Wang, Liu, Nugent (b30) 2023; 264 Zhang, Ma, Zhao (b40) 2021; 69 Sreelakshmy (b36) 2023; 3 I. Syed, I. Moinul, U. Mohiuddin, Q-Learning based particle swarm optimization algorithm for optimal path planning of swarm of mobile robots, in: Proceedings of 2019 International Conference on Advances in Science, Engineering and Robotics Technology, 2019, pp. 1–5. Zhang, Zheng, Cen (b1) 2005; 17 Xie, Qin, Zhou (b12) 2019; 186 Zhan, Sha, Xiao (b22) 2023; 9 Wang, Liu, Qin (b9) 2018; 37 Wang, Shan, Lu (b8) 2014; 64 Iiduka (b42) 2021; 52 Wang, Zhu, Zhu (b4) 2023; 59 Sutton (b32) 1988; 3 Zhang, Xia, Li (b16) 2022; 250 S. Li, X. Xu, L. Zuo, Dynamic path planning of a mobile robot with improved Q-learning algorithm, in: 2015 IEEE International Conference on Information and Automation, 2015, pp. 409–414. Zhou, Guo, Li (b10) 2021; 42 Zhou, Sun, Wu (b14) 2022; 42 Claussmann, Revilloud, Gruyer (b11) 2019; 21 Emmanouil (b35) 2016; 84 Li (b23) 2023; 53 Z. Bai, H. Pang, M. Liu, et al., An improved Q-Learning algorithm and its application to the optimized path planning for unmanned ground robot with obstacle avoidance, in: 2022 6th CAA International Conference on Vehicular Control and Intelligence, CVCI, 2022, pp. 1–6. Aleksandr, Yakovlev, Suvorov (b34) 2018; 123 Y.Z. Cong, S.G. Ponnambalam, Mobile robot path planning using ant colony optimization, in: 2009 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, 2009, pp. 851–856. Ma, Guan (b2) 2008; 3 Sombolestan, Rasooli, Khodaygan (b17) 2019; 10 Song, Zhou, Li (b27) 2023 Zhang, Lin, Chen (b5) 2018; 10 Hu, Chen, Tang (b7) 2018; 100 Yuan, Yu, Gu (b20) 2019; 175 greedy exploration in reinforcement learning based on value differences, in: Annual Conference on Artificial Intelligence, 2010, pp. 203–210. Zhou, Zhang, Wu (b15) 2022; 37 Low, Ong, Low, Omar (b21) 2022; 199 Xu, Yuan (b24) 2019; 27 C. Yan, X. Xiang, A path planning algorithm for UAV based on improved Q-learning, in: 2018 2nd International Conference on Robotics and Automation Sciences, ICRAS, 2018, pp. 1–5. Zhang, Li, Zhang (b25) 2021; 37 Wu, Low, Pang (b6) 2021; 70 Low, Ong, Cheah (b33) 2019; 115 Džeroski, Raedt, Driessens (b19) 2001; 43 M. Tokic, Adaptive Konar, Chakraborty, Singh (b18) 2013; 43 Gao, Ma, Liu (b26) 2018; 36 Li, Pan, Li (b41) 2023; 31 Liu, Tong (b13) 2021; 41 Tan, Zhang, Shi (b31) 2020; 18 10.1016/j.knosys.2024.111400_b29 Tan (10.1016/j.knosys.2024.111400_b31) 2020; 18 10.1016/j.knosys.2024.111400_b28 Aleksandr (10.1016/j.knosys.2024.111400_b34) 2018; 123 Liu (10.1016/j.knosys.2024.111400_b13) 2021; 41 Low (10.1016/j.knosys.2024.111400_b21) 2022; 199 Song (10.1016/j.knosys.2024.111400_b27) 2023 Sombolestan (10.1016/j.knosys.2024.111400_b17) 2019; 10 Li (10.1016/j.knosys.2024.111400_b23) 2023; 53 Zhan (10.1016/j.knosys.2024.111400_b22) 2023; 9 Wang (10.1016/j.knosys.2024.111400_b9) 2018; 37 Zhou (10.1016/j.knosys.2024.111400_b10) 2021; 42 10.1016/j.knosys.2024.111400_b3 Zhou (10.1016/j.knosys.2024.111400_b15) 2022; 37 Zhang (10.1016/j.knosys.2024.111400_b16) 2022; 250 Konar (10.1016/j.knosys.2024.111400_b18) 2013; 43 Sutton (10.1016/j.knosys.2024.111400_b32) 1988; 3 Zhang (10.1016/j.knosys.2024.111400_b5) 2018; 10 Wang (10.1016/j.knosys.2024.111400_b8) 2014; 64 Li (10.1016/j.knosys.2024.111400_b41) 2023; 31 Iiduka (10.1016/j.knosys.2024.111400_b42) 2021; 52 Claussmann (10.1016/j.knosys.2024.111400_b11) 2019; 21 Emmanouil (10.1016/j.knosys.2024.111400_b35) 2016; 84 10.1016/j.knosys.2024.111400_b38 10.1016/j.knosys.2024.111400_b39 Xie (10.1016/j.knosys.2024.111400_b12) 2019; 186 Ma (10.1016/j.knosys.2024.111400_b2) 2008; 3 Xu (10.1016/j.knosys.2024.111400_b24) 2019; 27 10.1016/j.knosys.2024.111400_b37 Zhang (10.1016/j.knosys.2024.111400_b1) 2005; 17 Hu (10.1016/j.knosys.2024.111400_b7) 2018; 100 Wang (10.1016/j.knosys.2024.111400_b30) 2023; 264 Zhang (10.1016/j.knosys.2024.111400_b40) 2021; 69 Yuan (10.1016/j.knosys.2024.111400_b20) 2019; 175 Džeroski (10.1016/j.knosys.2024.111400_b19) 2001; 43 Gao (10.1016/j.knosys.2024.111400_b26) 2018; 36 Wang (10.1016/j.knosys.2024.111400_b4) 2023; 59 Wu (10.1016/j.knosys.2024.111400_b6) 2021; 70 Zhang (10.1016/j.knosys.2024.111400_b25) 2021; 37 Zhou (10.1016/j.knosys.2024.111400_b14) 2022; 42 Sreelakshmy (10.1016/j.knosys.2024.111400_b36) 2023; 3 Low (10.1016/j.knosys.2024.111400_b33) 2019; 115 |
| References_xml | – volume: 41 start-page: 185 year: 2021 end-page: 190 ident: b13 article-title: Urban transportation path planning based on reinforcement learning publication-title: J. Comput. Appl. – volume: 52 start-page: 13250 year: 2021 end-page: 13261 ident: b42 article-title: Appropriate learning rates of adaptive learning rate optimization algorithms for training deep neural networks publication-title: IEEE Trans. Cybern. – volume: 3 start-page: 9 year: 1988 end-page: 44 ident: b32 article-title: Learning to predict by the methods of temporal difference publication-title: Mach. Learn. – volume: 53 start-page: 27110 year: 2023 end-page: 27127 ident: b23 article-title: INNES: An intelligent network penetration testing model based on deep reinforcement learning publication-title: Appl. Intell. – volume: 18 start-page: 53 year: 2020 end-page: 60 ident: b31 article-title: Reinforcement-learning-based multi-slot rail empty wagon real-time distribution publication-title: J. Transport. Eng. Inf. – volume: 43 start-page: 1141 year: 2013 end-page: 1153 ident: b18 article-title: A deterministic improved Q-learning for path planning of a mobile robot publication-title: IEEE Trans. Syst. Man Cybern.: Syst. – volume: 37 start-page: 5 year: 2018 end-page: 8 ident: b9 article-title: Review on intelligent path planning algorithm of mobile robots publication-title: Transducer Microsyst. Technol. – volume: 43 start-page: 7 year: 2001 end-page: 52 ident: b19 article-title: Relational reinforcement learning publication-title: Mach. Learn. – reference: C. Yan, X. Xiang, A path planning algorithm for UAV based on improved Q-learning, in: 2018 2nd International Conference on Robotics and Automation Sciences, ICRAS, 2018, pp. 1–5. – volume: 64 start-page: 1664 year: 2014 end-page: 1678 ident: b8 article-title: Real-time path planning based on hybrid-VANET-enhanced transportation system publication-title: IEEE Trans. Veh. Technol. – volume: 37 start-page: 1 year: 2021 end-page: 6 ident: b25 article-title: Local path planning of mobile robot based on improved Q-learning algorithm publication-title: J. Shandong Univ. Technol. (Nat. Sci. Ed.) – volume: 115 start-page: 143 year: 2019 end-page: 161 ident: b33 article-title: Solving the optimal path planning of a mobile robot using improved Q-learning publication-title: Robot. Auton. Syst. – volume: 21 start-page: 1826 year: 2019 end-page: 1848 ident: b11 article-title: A review of motion planning for highway autonomous driving publication-title: IEEE Trans. Intell. Transp. Syst. – volume: 84 start-page: 829 year: 2016 end-page: 858 ident: b35 article-title: A review of global path planning methods for occupancy grid maps regardless of obstacle density publication-title: J. Intell. Robot. Syst. – volume: 100 start-page: 482 year: 2018 end-page: 500 ident: b7 article-title: Dynamic path planning for autonomous driving on various roads with avoidance of static and moving obstacles publication-title: Mech. Syst. Signal Process. – volume: 37 start-page: 1314 year: 2022 end-page: 1322 ident: b15 article-title: An adaptive path planning algorithm for local delivery of confidential documents based on block-chain publication-title: J. Data Acquisit. Process. – volume: 10 start-page: 450 year: 2018 ident: b5 article-title: Path planning for the mobile robot: A review publication-title: Symmetry – volume: 36 start-page: 439 year: 2018 end-page: 443 ident: b26 article-title: Application of improved Q-learning algorithm in path planning publication-title: J. Jilin Univ. (Inf. Sci. Ed.) – volume: 9 start-page: 104 year: 2023 end-page: 120 ident: b22 article-title: Automated windows domain penetration method based on reinforcement learning publication-title: Chin. J. Netw. Inf. Secur. – volume: 250 year: 2022 ident: b16 article-title: Multi-objective particle swarm optimization with multi-mode collaboration based on reinforcement learning for path planning of unmanned air vehicles publication-title: Knowl.-Based Syst. – year: 2023 ident: b27 article-title: Research on path planning algorithm based on improved Q-learning algorithm publication-title: J. Chin. Comput. Syst. – volume: 175 start-page: 107 year: 2019 end-page: 117 ident: b20 article-title: A novel multi-step Q-learning method to improve data efficiency for deep reinforcement learning publication-title: Knowl.-Based Syst. – volume: 264 year: 2023 ident: b30 article-title: Mobile agent path planning under uncertain environment using reinforcement learning and probabilistic model checking publication-title: Knowl.-Based Syst. – volume: 59 start-page: 51 year: 2023 end-page: 66 ident: b4 article-title: Review of path planning algorithms for mobile robots publication-title: Comput. Eng. Appl. – volume: 42 start-page: 86 year: 2022 end-page: 98 ident: b14 article-title: A location privacy preservation scheme based on consortium block-chain in VANET publication-title: J. Nanjing Univ. Posts Telecommun. (Nat. Sci. Ed.) – volume: 3 start-page: 45 year: 2023 ident: b36 article-title: 3D path optimisation of unmanned aerial vehicles using q learning-controlled GWO-aoa publication-title: Comput. Syst. Sci. Eng. – volume: 199 year: 2022 ident: b21 article-title: Modified Q-learning with distance metric and virtual target on path planning of mobile robot publication-title: Expert Syst. Appl. – volume: 186 year: 2019 ident: b12 article-title: A novel test-cost-sensitive attribute reduction approach using the binary bat algorithm publication-title: Knowl.-Based Syst. – reference: Z. Bai, H. Pang, M. Liu, et al., An improved Q-Learning algorithm and its application to the optimized path planning for unmanned ground robot with obstacle avoidance, in: 2022 6th CAA International Conference on Vehicular Control and Intelligence, CVCI, 2022, pp. 1–6. – reference: S. Li, X. Xu, L. Zuo, Dynamic path planning of a mobile robot with improved Q-learning algorithm, in: 2015 IEEE International Conference on Information and Automation, 2015, pp. 409–414. – reference: Y.Z. Cong, S.G. Ponnambalam, Mobile robot path planning using ant colony optimization, in: 2009 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, 2009, pp. 851–856. – volume: 42 start-page: 498 year: 2021 end-page: 505 ident: b10 article-title: Path planning of UAV using guided enhancement Q-learning algorithm publication-title: Acta Aeronaut. Astronaut. Sinica – volume: 17 start-page: 439 year: 2005 end-page: 443 ident: b1 article-title: Present situation and future development of mobile robot path planning technology publication-title: J. Syst. Simul. – volume: 3 start-page: 22 year: 2008 end-page: 24 ident: b2 article-title: Summarization for present situation and future development of path planning technology publication-title: Mod. Mach. – volume: 70 start-page: 7464 year: 2021 end-page: 7479 ident: b6 article-title: Swarm-based 4D path planning for drone operations in urban environments publication-title: IEEE Trans. Veh. Technol. – volume: 69 start-page: 1537 year: 2021 end-page: 1547 ident: b40 article-title: A novel energy management strategy based on dual reward function Q-learning for fuel cell hybrid electric vehicle publication-title: IEEE Trans. Ind. Electron. – volume: 31 start-page: 23974 year: 2023 end-page: 23989 ident: b41 article-title: Advanced root mean square propagation with the warm-up algorithm for fiber coupling publication-title: Opt. Express – reference: I. Syed, I. Moinul, U. Mohiuddin, Q-Learning based particle swarm optimization algorithm for optimal path planning of swarm of mobile robots, in: Proceedings of 2019 International Conference on Advances in Science, Engineering and Robotics Technology, 2019, pp. 1–5. – volume: 123 start-page: 347 year: 2018 end-page: 353 ident: b34 article-title: Grid path planning with deep reinforcement learning: Preliminary results publication-title: Procedia Comput. Sci. – volume: 10 start-page: 1841 year: 2019 end-page: 1850 ident: b17 article-title: Optimal path-planning for mobile robots to find a hidden target in an unknown environment based on machine learning publication-title: J. Ambient Intell. Humaniz. Comput. – reference: -greedy exploration in reinforcement learning based on value differences, in: Annual Conference on Artificial Intelligence, 2010, pp. 203–210. – volume: 27 start-page: 314 year: 2019 end-page: 320 ident: b24 article-title: Path planning for mobile robot based on improved reinforcement learning algorithm publication-title: J. Chin. Inertial Technol. – reference: M. Tokic, Adaptive – volume: 175 start-page: 107 year: 2019 ident: 10.1016/j.knosys.2024.111400_b20 article-title: A novel multi-step Q-learning method to improve data efficiency for deep reinforcement learning publication-title: Knowl.-Based Syst. doi: 10.1016/j.knosys.2019.03.018 – volume: 17 start-page: 439 issue: 2 year: 2005 ident: 10.1016/j.knosys.2024.111400_b1 article-title: Present situation and future development of mobile robot path planning technology publication-title: J. Syst. Simul. – volume: 41 start-page: 185 issue: 1 year: 2021 ident: 10.1016/j.knosys.2024.111400_b13 article-title: Urban transportation path planning based on reinforcement learning publication-title: J. Comput. Appl. – ident: 10.1016/j.knosys.2024.111400_b28 doi: 10.1109/ICASERT.2019.8934450 – volume: 27 start-page: 314 issue: 3 year: 2019 ident: 10.1016/j.knosys.2024.111400_b24 article-title: Path planning for mobile robot based on improved reinforcement learning algorithm publication-title: J. Chin. Inertial Technol. – volume: 3 start-page: 22 year: 2008 ident: 10.1016/j.knosys.2024.111400_b2 article-title: Summarization for present situation and future development of path planning technology publication-title: Mod. Mach. – volume: 186 year: 2019 ident: 10.1016/j.knosys.2024.111400_b12 article-title: A novel test-cost-sensitive attribute reduction approach using the binary bat algorithm publication-title: Knowl.-Based Syst. doi: 10.1016/j.knosys.2019.104938 – volume: 43 start-page: 1141 issue: 5 year: 2013 ident: 10.1016/j.knosys.2024.111400_b18 article-title: A deterministic improved Q-learning for path planning of a mobile robot publication-title: IEEE Trans. Syst. Man Cybern.: Syst. doi: 10.1109/TSMCA.2012.2227719 – volume: 21 start-page: 1826 issue: 5 year: 2019 ident: 10.1016/j.knosys.2024.111400_b11 article-title: A review of motion planning for highway autonomous driving publication-title: IEEE Trans. Intell. Transp. Syst. doi: 10.1109/TITS.2019.2913998 – volume: 264 year: 2023 ident: 10.1016/j.knosys.2024.111400_b30 article-title: Mobile agent path planning under uncertain environment using reinforcement learning and probabilistic model checking publication-title: Knowl.-Based Syst. doi: 10.1016/j.knosys.2023.110355 – volume: 37 start-page: 1314 issue: 6 year: 2022 ident: 10.1016/j.knosys.2024.111400_b15 article-title: An adaptive path planning algorithm for local delivery of confidential documents based on block-chain publication-title: J. Data Acquisit. Process. – volume: 10 start-page: 450 issue: 10 year: 2018 ident: 10.1016/j.knosys.2024.111400_b5 article-title: Path planning for the mobile robot: A review publication-title: Symmetry doi: 10.3390/sym10100450 – volume: 37 start-page: 5 issue: 8 year: 2018 ident: 10.1016/j.knosys.2024.111400_b9 article-title: Review on intelligent path planning algorithm of mobile robots publication-title: Transducer Microsyst. Technol. – volume: 84 start-page: 829 year: 2016 ident: 10.1016/j.knosys.2024.111400_b35 article-title: A review of global path planning methods for occupancy grid maps regardless of obstacle density publication-title: J. Intell. Robot. Syst. doi: 10.1007/s10846-016-0362-z – volume: 9 start-page: 104 issue: 4 year: 2023 ident: 10.1016/j.knosys.2024.111400_b22 article-title: Automated windows domain penetration method based on reinforcement learning publication-title: Chin. J. Netw. Inf. Secur. – ident: 10.1016/j.knosys.2024.111400_b3 doi: 10.1109/AIM.2009.5229903 – volume: 53 start-page: 27110 issue: 22 year: 2023 ident: 10.1016/j.knosys.2024.111400_b23 article-title: INNES: An intelligent network penetration testing model based on deep reinforcement learning publication-title: Appl. Intell. doi: 10.1007/s10489-023-04946-1 – volume: 250 year: 2022 ident: 10.1016/j.knosys.2024.111400_b16 article-title: Multi-objective particle swarm optimization with multi-mode collaboration based on reinforcement learning for path planning of unmanned air vehicles publication-title: Knowl.-Based Syst. doi: 10.1016/j.knosys.2022.109075 – volume: 199 year: 2022 ident: 10.1016/j.knosys.2024.111400_b21 article-title: Modified Q-learning with distance metric and virtual target on path planning of mobile robot publication-title: Expert Syst. Appl. doi: 10.1016/j.eswa.2022.117191 – volume: 100 start-page: 482 year: 2018 ident: 10.1016/j.knosys.2024.111400_b7 article-title: Dynamic path planning for autonomous driving on various roads with avoidance of static and moving obstacles publication-title: Mech. Syst. Signal Process. doi: 10.1016/j.ymssp.2017.07.019 – volume: 3 start-page: 45 issue: 3 year: 2023 ident: 10.1016/j.knosys.2024.111400_b36 article-title: 3D path optimisation of unmanned aerial vehicles using q learning-controlled GWO-aoa publication-title: Comput. Syst. Sci. Eng. – ident: 10.1016/j.knosys.2024.111400_b29 doi: 10.1109/CVCI56766.2022.9964859 – volume: 52 start-page: 13250 issue: 12 year: 2021 ident: 10.1016/j.knosys.2024.111400_b42 article-title: Appropriate learning rates of adaptive learning rate optimization algorithms for training deep neural networks publication-title: IEEE Trans. Cybern. doi: 10.1109/TCYB.2021.3107415 – volume: 3 start-page: 9 issue: 1 year: 1988 ident: 10.1016/j.knosys.2024.111400_b32 article-title: Learning to predict by the methods of temporal difference publication-title: Mach. Learn. doi: 10.1023/A:1022633531479 – ident: 10.1016/j.knosys.2024.111400_b39 doi: 10.1109/ICRAS.2018.8443226 – volume: 42 start-page: 498 issue: 9 year: 2021 ident: 10.1016/j.knosys.2024.111400_b10 article-title: Path planning of UAV using guided enhancement Q-learning algorithm publication-title: Acta Aeronaut. Astronaut. Sinica – year: 2023 ident: 10.1016/j.knosys.2024.111400_b27 article-title: Research on path planning algorithm based on improved Q-learning algorithm publication-title: J. Chin. Comput. Syst. – volume: 18 start-page: 53 issue: 4 year: 2020 ident: 10.1016/j.knosys.2024.111400_b31 article-title: Reinforcement-learning-based multi-slot rail empty wagon real-time distribution publication-title: J. Transport. Eng. Inf. – volume: 37 start-page: 1 issue: 2 year: 2021 ident: 10.1016/j.knosys.2024.111400_b25 article-title: Local path planning of mobile robot based on improved Q-learning algorithm publication-title: J. Shandong Univ. Technol. (Nat. Sci. Ed.) – volume: 59 start-page: 51 issue: 20 year: 2023 ident: 10.1016/j.knosys.2024.111400_b4 article-title: Review of path planning algorithms for mobile robots publication-title: Comput. Eng. Appl. – volume: 123 start-page: 347 year: 2018 ident: 10.1016/j.knosys.2024.111400_b34 article-title: Grid path planning with deep reinforcement learning: Preliminary results publication-title: Procedia Comput. Sci. doi: 10.1016/j.procs.2018.01.054 – ident: 10.1016/j.knosys.2024.111400_b37 doi: 10.1007/978-3-642-16111-7_23 – volume: 10 start-page: 1841 year: 2019 ident: 10.1016/j.knosys.2024.111400_b17 article-title: Optimal path-planning for mobile robots to find a hidden target in an unknown environment based on machine learning publication-title: J. Ambient Intell. Humaniz. Comput. doi: 10.1007/s12652-018-0777-4 – volume: 43 start-page: 7 year: 2001 ident: 10.1016/j.knosys.2024.111400_b19 article-title: Relational reinforcement learning publication-title: Mach. Learn. doi: 10.1023/A:1007694015589 – ident: 10.1016/j.knosys.2024.111400_b38 doi: 10.1109/ICInfA.2015.7279322 – volume: 31 start-page: 23974 issue: 15 year: 2023 ident: 10.1016/j.knosys.2024.111400_b41 article-title: Advanced root mean square propagation with the warm-up algorithm for fiber coupling publication-title: Opt. Express doi: 10.1364/OE.494734 – volume: 70 start-page: 7464 issue: 8 year: 2021 ident: 10.1016/j.knosys.2024.111400_b6 article-title: Swarm-based 4D path planning for drone operations in urban environments publication-title: IEEE Trans. Veh. Technol. doi: 10.1109/TVT.2021.3093318 – volume: 69 start-page: 1537 issue: 2 year: 2021 ident: 10.1016/j.knosys.2024.111400_b40 article-title: A novel energy management strategy based on dual reward function Q-learning for fuel cell hybrid electric vehicle publication-title: IEEE Trans. Ind. Electron. doi: 10.1109/TIE.2021.3062273 – volume: 42 start-page: 86 issue: 6 year: 2022 ident: 10.1016/j.knosys.2024.111400_b14 article-title: A location privacy preservation scheme based on consortium block-chain in VANET publication-title: J. Nanjing Univ. Posts Telecommun. (Nat. Sci. Ed.) – volume: 115 start-page: 143 year: 2019 ident: 10.1016/j.knosys.2024.111400_b33 article-title: Solving the optimal path planning of a mobile robot using improved Q-learning publication-title: Robot. Auton. Syst. doi: 10.1016/j.robot.2019.02.013 – volume: 64 start-page: 1664 issue: 5 year: 2014 ident: 10.1016/j.knosys.2024.111400_b8 article-title: Real-time path planning based on hybrid-VANET-enhanced transportation system publication-title: IEEE Trans. Veh. Technol. doi: 10.1109/TVT.2014.2335201 – volume: 36 start-page: 439 issue: 4 year: 2018 ident: 10.1016/j.knosys.2024.111400_b26 article-title: Application of improved Q-learning algorithm in path planning publication-title: J. Jilin Univ. (Inf. Sci. Ed.) |
| SSID | ssj0002218 |
| Score | 2.6338198 |
| Snippet | The Q-Learning algorithm is a reinforcement learning technique widely used in various fields such as path planning, intelligent transportation, penetration... |
| SourceID | crossref elsevier |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 111400 |
| SubjectTerms | Adaptive learning rate Local path planning Mobile robot Q-Learning algorithm Reinforcement learning |
| Title | An optimized Q-Learning algorithm for mobile robot local path planning |
| URI | https://dx.doi.org/10.1016/j.knosys.2024.111400 |
| Volume | 286 |
| WOSCitedRecordID | wos001170720800001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 customDbUrl: eissn: 1872-7409 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0002218 issn: 0950-7051 databaseCode: AIEXJ dateStart: 19950201 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1Lb9QwELag5cCFN6IUkA_cKldO4sT2cVW1glJVVBRp6SVyYodN202qbRa1_HrGj2QjLeIlcYkSy46TmU_jz6PxDEJvEyZ0woUhTKcJYbKMiDBRYV0CPFWap7GhrtgEPz4W06n8GMqlXrtyArxpxM2NvPqvqoY2ULY9OvsX6h5eCg1wD0qHK6gdrn-k-AkQQLAD8_o7cMkTctT7PtTl13ZRd7O5iyyctwXYg51FW7TdjlvQbIrVma0q7YoYjUnrh97vRuyap0P254GMn83apdXUyQhpR7V3rH5R4VXW7rteh7W6HTWezVyrDa69XZqxDyJmozPdzjG2djgmeBgp4TTkkzXevgoOhJ5ROTbAsU-GvWbMvV_hfPeiaeHHdu3E1sIzSleL1xBS-MlOZ2ezQbE2yetdtBnzVIKl25y8358eDutzHDuv7_B5_YFKF_W3PtfPCcuIhJw-Qg_C7gFPvNYfozumeYIe9pU5cDDUT9HBpMEDCPAKBHgAAQYQYA8C7ECAHQiwBQHuQfAMfT7YP917R0LFDFLC1q8jmlYiZVlRMKGoYlorEWVSMaNiChvHuCjTSmnBKyGjKtEM9t6UGXguykwzUyXP0UbTNuYFwkZroVSVGCkrZoCEK-gEbMZkqcyKqNpCSS-WvAzp5G1Vk8u8jxs8z70wcyvM3AtzC5Fh1JVPp_Kb_ryXeB4ooad6OYDklyNf_vPIbXR_hfFXaKNbLM1rdK_81tXXizcBTT8A-KqH6Q |
| linkProvider | Elsevier |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=An+optimized+Q-Learning+algorithm+for+mobile+robot+local+path+planning&rft.jtitle=Knowledge-based+systems&rft.au=Zhou%2C+Qian&rft.au=Lian%2C+Yang&rft.au=Wu%2C+Jiayang&rft.au=Zhu%2C+Mengyue&rft.date=2024-02-28&rft.pub=Elsevier+B.V&rft.issn=0950-7051&rft.eissn=1872-7409&rft.volume=286&rft_id=info:doi/10.1016%2Fj.knosys.2024.111400&rft.externalDocID=S0950705124000352 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0950-7051&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0950-7051&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0950-7051&client=summon |