Variable Action Set-Based Monte Carlo Tree Search Algorithm for AAV Autonomous Collision Avoidance

Autonomous collision avoidance of Autonomous Aerial Vehicles (AAVs) is becoming a hot research topic in Urban Air Mobility (UAM) operations. Many studies modeled this problem as a Markov decision process (MDP) and adopted intelligent methods such as reinforcement learning and Monte Carlo Tree Search...

Full description

Saved in:
Bibliographic Details
Published in:IEEE access Vol. 13; pp. 43861 - 43877
Main Authors: Xu, Fulong, Li, Bo
Format: Journal Article
Language:English
Published: IEEE 2025
Subjects:
AAV
Aircraft
Atmospheric modeling
Autonomous aerial vehicles
Autonomous collision avoidance
Classification algorithms
Collision avoidance
Energy efficiency
Markov decision process
Monte Carlo methods
Monte Carlo tree search
Real-time systems
Trajectory
Turning
variable action set
ISSN:2169-3536, 2169-3536
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Autonomous collision avoidance of Autonomous Aerial Vehicles (AAVs) is becoming a hot research topic in Urban Air Mobility (UAM) operations. Many studies modeled this problem as a Markov decision process (MDP) and adopted intelligent methods such as reinforcement learning and Monte Carlo Tree Search (MCTS) to design collision avoidance algorithms. Among these studies, some algorithms with discrete action sets have achieved good results in collision avoidance. However, the fixed discrete action set often leads to low flight quality, as the rigid control makes the AAV unable to stably fly straight in the desired direction but can only fly wobbly. To address this problem, a Variable Action Set-based MCTS algorithm (VASM) for autonomous collision avoidance is proposed in this paper. By introducing an additional action option that is carefully designed to improve the stability and efficiency of flight along any desired heading, a variable action set is constructed for VASM. In addition, a reward function that takes into account more rational evaluation factors is also proposed to facilitate better decision-making. We conducted numerical experiments to verify the effectiveness of VASM, and results show that compared with the conventional MCTS algorithm, VASM can better improve flight control and flight quality with only a small increase in computational cost, achieving more efficient and energy-saving flight guidance. Specifically, in high-density airspace, VASM reduces average flight distance by 15.34%, average flight time by 15.08%, average flight turns by 36.64%, conflict probability by 18.56%, while increasing target arrival probability to 98.3%.
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2025.3547885