Variable Action Set-Based Monte Carlo Tree Search Algorithm for AAV Autonomous Collision Avoidance
Autonomous collision avoidance of Autonomous Aerial Vehicles (AAVs) is becoming a hot research topic in Urban Air Mobility (UAM) operations. Many studies modeled this problem as a Markov decision process (MDP) and adopted intelligent methods such as reinforcement learning and Monte Carlo Tree Search...
Gespeichert in:
| Veröffentlicht in: | IEEE access Jg. 13; S. 43861 - 43877 |
|---|---|
| Hauptverfasser: | , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
IEEE
2025
|
| Schlagworte: | |
| ISSN: | 2169-3536, 2169-3536 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | Autonomous collision avoidance of Autonomous Aerial Vehicles (AAVs) is becoming a hot research topic in Urban Air Mobility (UAM) operations. Many studies modeled this problem as a Markov decision process (MDP) and adopted intelligent methods such as reinforcement learning and Monte Carlo Tree Search (MCTS) to design collision avoidance algorithms. Among these studies, some algorithms with discrete action sets have achieved good results in collision avoidance. However, the fixed discrete action set often leads to low flight quality, as the rigid control makes the AAV unable to stably fly straight in the desired direction but can only fly wobbly. To address this problem, a Variable Action Set-based MCTS algorithm (VASM) for autonomous collision avoidance is proposed in this paper. By introducing an additional action option that is carefully designed to improve the stability and efficiency of flight along any desired heading, a variable action set is constructed for VASM. In addition, a reward function that takes into account more rational evaluation factors is also proposed to facilitate better decision-making. We conducted numerical experiments to verify the effectiveness of VASM, and results show that compared with the conventional MCTS algorithm, VASM can better improve flight control and flight quality with only a small increase in computational cost, achieving more efficient and energy-saving flight guidance. Specifically, in high-density airspace, VASM reduces average flight distance by 15.34%, average flight time by 15.08%, average flight turns by 36.64%, conflict probability by 18.56%, while increasing target arrival probability to 98.3%. |
|---|---|
| AbstractList | Autonomous collision avoidance of Autonomous Aerial Vehicles (AAVs) is becoming a hot research topic in Urban Air Mobility (UAM) operations. Many studies modeled this problem as a Markov decision process (MDP) and adopted intelligent methods such as reinforcement learning and Monte Carlo Tree Search (MCTS) to design collision avoidance algorithms. Among these studies, some algorithms with discrete action sets have achieved good results in collision avoidance. However, the fixed discrete action set often leads to low flight quality, as the rigid control makes the AAV unable to stably fly straight in the desired direction but can only fly wobbly. To address this problem, a Variable Action Set-based MCTS algorithm (VASM) for autonomous collision avoidance is proposed in this paper. By introducing an additional action option that is carefully designed to improve the stability and efficiency of flight along any desired heading, a variable action set is constructed for VASM. In addition, a reward function that takes into account more rational evaluation factors is also proposed to facilitate better decision-making. We conducted numerical experiments to verify the effectiveness of VASM, and results show that compared with the conventional MCTS algorithm, VASM can better improve flight control and flight quality with only a small increase in computational cost, achieving more efficient and energy-saving flight guidance. Specifically, in high-density airspace, VASM reduces average flight distance by 15.34%, average flight time by 15.08%, average flight turns by 36.64%, conflict probability by 18.56%, while increasing target arrival probability to 98.3%. |
| Author | Li, Bo Xu, Fulong |
| Author_xml | – sequence: 1 givenname: Fulong orcidid: 0000-0002-3823-4844 surname: Xu fullname: Xu, Fulong email: xufulong@cafuc.edu.cn organization: College of Air Traffic Management, Civil Aviation Flight University of China, Guanghan, China – sequence: 2 givenname: Bo orcidid: 0009-0005-7891-3189 surname: Li fullname: Li, Bo organization: College of Air Traffic Management, Civil Aviation Flight University of China, Guanghan, China |
| BookMark | eNpNkNtKxDAQhoMoeNon0Iu8QNccmqS5rMUTKF6s7m2YpFONdBtJq-Db23VFnJsZZvi_ge-Y7A9pQELOOFtyzuxF3TRXq9VSMKGWUpWmqtQeORJc20Iqqff_zYdkMY5vbK5qXilzRPwacgTfI63DFNNAVzgVlzBiSx_SMCFtIPeJPmXE-QQ5vNK6f0k5Tq8b2qVM63pN648pDWmTPkbapL6P4xZUf6bYwhDwlBx00I-4-O0n5Pn66qm5Le4fb-6a-r4IUsqpQMlRGGWMaCtZ2o5B8FwbMKYVVScqYVF3UHJQOjAvwCLTXGrUKCBYaeQJudtx2wRv7j3HDeQvlyC6n0XKLw7yFEOPrhMMvbed116WLdNWS0Ql53feM6vYzJI7VshpHDN2fzzO3Na621l3W-vu1_qcOt-lIiL-S1hmS23kN2G_f1U |
| CODEN | IAECCG |
| Cites_doi | 10.1016/j.trc.2011.08.008 10.2514/3.20934 10.1016/j.cja.2014.12.031 10.1109/ICRA.2011.5980246 10.1016/j.cja.2023.09.033 10.2514/1.27553 10.2514/6.2018-3365 10.1145/3374587.3374599 10.3390/drones8020060 10.1016/j.asoc.2023.110773 10.2514/atcq.23.2-3.137 10.1109/ICRA.2012.6225009 10.1109/6979.898217 10.2514/6.2004-5141 10.2514/1.D0120 10.2514/1.11168 10.1016/j.paerosci.2015.01.001 10.1109/ACCESS.2022.3201962 10.1016/j.physa.2023.129423 10.1109/ACCESS.2019.2909530 10.3390/drones8050205 10.1109/ACC.2008.4587222 10.23919/ECC.2001.7076321 10.1109/TITS.2020.3048360 10.1109/ACC.2002.1023918 10.1109/CDC.2009.5400520 10.2514/1.G000218 10.1109/IROS.2012.6385823 10.1109/ICUAS.2016.7502621 10.2514/1.48475 10.1109/SICE.2008.4655013 10.1016/j.paerosci.2020.100640 10.1109/ACC.2007.4283147 10.2514/2.4678 10.23919/ChiCC.2017.8027884 10.1109/CDC.2003.1271710 10.2514/6.2010-8040 10.1016/j.neunet.2023.02.027 |
| ContentType | Journal Article |
| DBID | 97E ESBDL RIA RIE AAYXX CITATION DOA |
| DOI | 10.1109/ACCESS.2025.3547885 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE Xplore Open Access Journals IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef DOAJ: Directory of Open Access Journals |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: DOA name: DOAJ : Directory of Open Access Journals [open access] url: https://www.doaj.org/ sourceTypes: Open Website – sequence: 2 dbid: RIE name: IEEE/IET Electronic Library (IEL) (UW System Shared) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 2169-3536 |
| EndPage | 43877 |
| ExternalDocumentID | oai_doaj_org_article_f20ebb9fb6b34d06963ee53167bb0950 10_1109_ACCESS_2025_3547885 10909467 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: General Program of Civil Aviation Flight University of China grantid: XJ2024016301 funderid: 10.13039/501100002881 |
| GroupedDBID | 0R~ 4.4 5VS 6IK 97E AAJGR ABAZT ABVLG ACGFS ADBBV AGSQL ALMA_UNASSIGNED_HOLDINGS BCNDV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD ESBDL GROUPED_DOAJ IPLJI JAVBF KQ8 M43 M~E O9- OCL OK1 RIA RIE RNS AAYXX CITATION |
| ID | FETCH-LOGICAL-c333t-e31e275772d8349f0acb167a77d28f2829e6fa41a56c0b2a9e06136e6e2ac9373 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 0 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001445086900021&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2169-3536 |
| IngestDate | Fri Oct 03 12:49:25 EDT 2025 Sat Nov 29 08:12:38 EST 2025 Wed Aug 27 01:42:30 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Language | English |
| License | https://creativecommons.org/licenses/by/4.0/legalcode |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c333t-e31e275772d8349f0acb167a77d28f2829e6fa41a56c0b2a9e06136e6e2ac9373 |
| ORCID | 0000-0002-3823-4844 0009-0005-7891-3189 |
| OpenAccessLink | https://ieeexplore.ieee.org/document/10909467 |
| PageCount | 17 |
| ParticipantIDs | crossref_primary_10_1109_ACCESS_2025_3547885 ieee_primary_10909467 doaj_primary_oai_doaj_org_article_f20ebb9fb6b34d06963ee53167bb0950 |
| PublicationCentury | 2000 |
| PublicationDate | 20250000 2025-00-00 2025-01-01 |
| PublicationDateYYYYMMDD | 2025-01-01 |
| PublicationDate_xml | – year: 2025 text: 20250000 |
| PublicationDecade | 2020 |
| PublicationTitle | IEEE access |
| PublicationTitleAbbrev | Access |
| PublicationYear | 2025 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| References | ref13 ref35 ref12 ref34 ref15 ref14 ref36 ref31 ref30 ref11 ref33 ref10 ref2 ref1 Yang (ref39); 8 ref17 ref16 ref19 ref18 Lia (ref37) 2020; 41 Kochenderfer (ref32) 2012; 19 (ref46) 2017 ref24 ref23 ref45 ref26 ref25 ref20 ref42 ref41 ref22 Delahaye (ref28) ref44 Pham (ref4) ref21 ref43 Kocsis (ref38) 2006 ref27 ref29 ref8 ref7 ref9 (ref40) 2018 ref3 ref6 ref5 |
| References_xml | – volume: 41 issue: 8 year: 2020 ident: ref37 article-title: Collision avoidance decision method for UAVs in random search combined with jump point guidance publication-title: Acta Aeronautica et Astronautica Sinica – ident: ref9 doi: 10.1016/j.trc.2011.08.008 – ident: ref17 doi: 10.2514/3.20934 – ident: ref12 doi: 10.1016/j.cja.2014.12.031 – volume: 8 start-page: 1 volume-title: Proc. Int. Conf. Res. Air Transp. (ICRAT) ident: ref39 article-title: Autonomous on-demand free flight operations in urban air mobility using Monte Carlo tree search – ident: ref29 doi: 10.1109/ICRA.2011.5980246 – ident: ref34 doi: 10.1016/j.cja.2023.09.033 – ident: ref24 doi: 10.2514/1.27553 – ident: ref44 doi: 10.2514/6.2018-3365 – ident: ref2 doi: 10.1145/3374587.3374599 – ident: ref36 doi: 10.3390/drones8020060 – ident: ref33 doi: 10.1016/j.asoc.2023.110773 – ident: ref45 doi: 10.2514/atcq.23.2-3.137 – ident: ref21 doi: 10.1109/ICRA.2012.6225009 – ident: ref7 doi: 10.1109/6979.898217 – ident: ref8 doi: 10.2514/6.2004-5141 – ident: ref1 doi: 10.2514/1.D0120 – ident: ref16 doi: 10.2514/1.11168 – ident: ref6 doi: 10.1016/j.paerosci.2015.01.001 – ident: ref43 doi: 10.1109/ACCESS.2022.3201962 – volume-title: Near Midair Collision Reporting year: 2017 ident: ref46 – ident: ref11 doi: 10.1016/j.physa.2023.129423 – ident: ref27 doi: 10.1109/ACCESS.2019.2909530 – ident: ref42 doi: 10.3390/drones8050205 – ident: ref13 doi: 10.1109/ACC.2008.4587222 – ident: ref18 doi: 10.23919/ECC.2001.7076321 – start-page: 1 volume-title: Proc. 13th USA/Europe Air Traffic Manage. Res. Develop. Seminar ident: ref4 article-title: A machine learning approach for conflict resolution in dense traffic scenarios with uncertainties – ident: ref5 doi: 10.1109/TITS.2020.3048360 – ident: ref19 doi: 10.1109/ACC.2002.1023918 – ident: ref20 doi: 10.1109/CDC.2009.5400520 – start-page: 71 volume-title: Proc. EIWAC, 2nd ENRI Int. Workshop ATM/CNS ident: ref28 article-title: Aircraft conflict resolution by genetic algorithm and B-spline approximation – ident: ref23 doi: 10.2514/1.G000218 – ident: ref22 doi: 10.1109/IROS.2012.6385823 – ident: ref26 doi: 10.1109/ICUAS.2016.7502621 – ident: ref30 doi: 10.2514/1.48475 – volume: 19 start-page: 17 issue: 1 year: 2012 ident: ref32 article-title: Next generation airborne collision avoidance system publication-title: Lincoln Lab. J. – year: 2006 ident: ref38 article-title: Improved Monte-Carlo search – volume-title: Acubed Vahana year: 2018 ident: ref40 – ident: ref10 doi: 10.1109/SICE.2008.4655013 – ident: ref41 doi: 10.1016/j.paerosci.2020.100640 – ident: ref14 doi: 10.1109/ACC.2007.4283147 – ident: ref25 doi: 10.2514/2.4678 – ident: ref3 doi: 10.23919/ChiCC.2017.8027884 – ident: ref15 doi: 10.1109/CDC.2003.1271710 – ident: ref31 doi: 10.2514/6.2010-8040 – ident: ref35 doi: 10.1016/j.neunet.2023.02.027 |
| SSID | ssj0000816957 |
| Score | 2.3344834 |
| Snippet | Autonomous collision avoidance of Autonomous Aerial Vehicles (AAVs) is becoming a hot research topic in Urban Air Mobility (UAM) operations. Many studies... |
| SourceID | doaj crossref ieee |
| SourceType | Open Website Index Database Publisher |
| StartPage | 43861 |
| SubjectTerms | AAV Aircraft Atmospheric modeling Autonomous aerial vehicles Autonomous collision avoidance Classification algorithms Collision avoidance Energy efficiency Markov decision process Monte Carlo methods Monte Carlo tree search Real-time systems Trajectory Turning variable action set |
| SummonAdditionalLinks | – databaseName: DOAJ: Directory of Open Access Journals dbid: DOA link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV07T8MwELZQxQAD4lFEeckDI6Gp7Tj2mFZUDFAhUapukR8XqAQtCml_P7YTUJhYWKMocb473d3n-L5D6MoYLrgpWORCIESMWhEpQkRkTFwkilBLQof37D6dTMR8Lh9bo778mbBaHrgGrl-QGLSWheaaMhtz5zAAie_f1tqVB4Gtx6lskakQg8WAyyRtZIYGsexno5H7IkcISXJDvYiVn57cSkVBsf_XiJWQYcb7aK8pDXFWL-kAbcHyEO22BAOPkJ45auubnXAW-hHwE1TR0CUiix-8zBQeqfJthaclAK4PEuPs7WVVLqrXd-zKU5xlM5ytK9_J4Cg_9tsGobkcZ5vVwnoP6KLn8e10dBc1UxIiQymtIqADIGniqmQrKJNFrIx26Kg0tUQU_kcp8EKxgUq4iTVREnwK58CBKOOKE3qMOsvVEk4QduxEEGalMsYwJbiywEC5CGASWijBeuj6G7D8oxbDyAOJiGVe45t7fPMG3x4aelB_bvVK1uGCs2_e2Df_y7491PUmab1POkbK09P_ePgZ2vELrrdUzlGnKtdwgbbNplp8lpfBpb4Aj-jNZA priority: 102 providerName: Directory of Open Access Journals |
| Title | Variable Action Set-Based Monte Carlo Tree Search Algorithm for AAV Autonomous Collision Avoidance |
| URI | https://ieeexplore.ieee.org/document/10909467 https://doaj.org/article/f20ebb9fb6b34d06963ee53167bb0950 |
| Volume | 13 |
| WOSCitedRecordID | wos001445086900021&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAON databaseName: DOAJ : Directory of Open Access Journals [open access] customDbUrl: eissn: 2169-3536 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000816957 issn: 2169-3536 databaseCode: DOA dateStart: 20130101 isFulltext: true titleUrlDefault: https://www.doaj.org/ providerName: Directory of Open Access Journals – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2169-3536 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000816957 issn: 2169-3536 databaseCode: M~E dateStart: 20130101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV27TsMwFLUAMcDAG1EelQdGAqntOM4YKhADICSgYov8uIFK0KCQMvLt-DqhKgMDSxRFluL4OL4P-5xLyLG1UklbisgvgRAJ7lSkGVORtXGZaMYdCwzv0XV6e6uenrK7jqweuDAAEA6fwSnehr18V9kppsrO8BAhloRfJItpKluy1iyhghUksiTtlIV807N8OPQf4WNAlpxy1K3Cgslz1ieI9P-qqhKMyuX6P7uzQdY675HmLdybZAEmW2R1TlNwm5iRj36RD0XzQFmg99BE595WOXqDSlR0qOvXij7UALQ9a0zz1-eqHjcvb9R7sDTPRzSfNkh2qKYfFDMLgX9O889q7HCS7JDHy4uH4VXUFVKILOe8iYAPgKWJd6Sd4iIrY23NQKY6TR1TJe6lgiy1GOhE2tgwnQFaeQkSmLbef-G7ZGlSTWCPUB_AKCZcpq21QiupHQjQfpGwCS-1Ej1y8jPAxXurl1GEOCPOihaPAvEoOjx65BxBmDVFsevwwA900f07RcliMCYrjTRcuFj6NQMgQQq_Md5DjHtkB8GZe1-Ly_4fzw_ICvahTaQckqWmnsIRWbafzfij7oeo3F9vvi76YYZ9A6CxzSs |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1Nb9QwEB1BQSocKNBWLLTgA0fSZm3HcY7pqlWrbldILKveIn9M2pXaDUqz_f14nFAtBw7cIstSHD_H82G_NwBfnVNauVomYQvERAqvE8O5TpxL68xw4XlkeC-m-Wymr6-L7wNZPXJhEDFePsMjeoxn-b5xa0qVHdMlQioJ_xxeUOmsga71lFKhGhJFlg_aQqHzcTmZhM8IUSDPjgQpV1HJ5A37E2X6_6qrEs3K2c5_DugtvBn8R1b2gL-DZ7h6D683VAV3wS5C_EuMKFZG0gL7gV1yEqyVZ1ekRcUmpr1r2LxFZP1tY1be3TTtsru9Z8GHZWW5YOW6I7pDs35glFuIDHRWPjZLT8tkD36enc4n58lQSiFxQoguQTFGnmfBlfZayKJOjbNjlZs891zXdJqKqjZybDLlUstNgWTnFSrkxgUPRuzD1qpZ4QdgIYTRXPrCOOek0cp4lGjCNuEyURstR_DtzwRXv3rFjCpGGmlR9XhUhEc14DGCEwLhqSvJXceGMNHV8PdUNU_R2qK2ygrpUxV2DcSMSPzWBh8xHcEegbPxvh6Xj_9o_wLb5_OraTW9mF1-glc0nj6tcgBbXbvGQ3jpHrvlQ_s5rrDfzY7OTg |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Variable+Action+Set-Based+Monte+Carlo+Tree+Search+Algorithm+for+AAV+Autonomous+Collision+Avoidance&rft.jtitle=IEEE+access&rft.au=Xu%2C+Fulong&rft.au=Li%2C+Bo&rft.date=2025&rft.issn=2169-3536&rft.eissn=2169-3536&rft.volume=13&rft.spage=43861&rft.epage=43877&rft_id=info:doi/10.1109%2FACCESS.2025.3547885&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_ACCESS_2025_3547885 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2169-3536&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2169-3536&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2169-3536&client=summon |