Distributed Safe Multi-Agent Reinforcement Learning: Joint Design of THz-Enabled UAV Trajectory and Channel Allocation - Termehchi, Atefeh; Syed, Aisha; Kennedy, William Sean; Erol-Kantarci, Melike (IEEE transactions on vehicular technology, 2024)
6G is anticipated to play a foundational role in realizing various emerging entertainment applications and critical societal services, such as smart agriculture, public safety, and so on. Providing the underlying communications for these applications will substantially increase demand for data rate, reliability, and resiliency. Given these requirements, Terahertz (THz)-enabled unmanned aerial vehicles (UAVs) are expected to support the essential functions within the future 6G networks. Meanwhile, due to the dynamic environment in a THz-enabled UAV-assisted network with multiple UAVs, using model-free multi-agent deep reinforcement learning (MADRL) becomes a promising approach to optimize scarce resources. However, satisfying cooperative safety constraints is not guaranteed in most previous MADRL techniques. Consequently, this paper aims to address the cooperative safety guarantee challenge in the joint design problem of UAV trajectory and channel allocation in a THz-enabled UAV-assisted network. Specifically, the goal of the proposed scheme is to maximize energy efficiency, considering the quality of service of each IoT device, the speed limitation of UAVs, and the collision avoidance constraint between UAVs. The problem is mixed integer nonlinear programming (MINP), known as NP-hard to solve. Thus, we propose a novel distributed safe MADRL (DSMADRL) approach for the safe trajectory design of UAVs using a data-driven uniformly ultimate boundedness stability method. Moreover, we theoretically show that the DSMADRL approach guarantees satisfying the cooperative safety constraint of collision avoidance between UAVs as well as maximizing energy efficiency. Furthermore, the matching method is used to allocate THz sub-bands distributively. Finally, the effectiveness of the algorithms is evaluated by comparing them with the existing multi-agent RL algorithms.