Distributed Beamforming Techniques for Cell-Free Wireless Networks Using Deep Reinforcement Learning

In a cell-free network, a large number of mobile devices are served simultaneously by several base stations (BSs)/access points(APs) using the same time/frequency resources. However, this creates high signal processing demands (e.g., for beamforming) at the transmitters and receivers. In this work,...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transactions on cognitive communications and networking Ročník 8; číslo 2; s. 1186 - 1201
Hlavní autoři: Fredj, Firas, Al-Eryani, Yasser, Maghsudi, Setareh, Akrout, Mohamed, Hossain, Ekram
Médium: Journal Article
Jazyk:angličtina
Vydáno: Piscataway IEEE 01.06.2022
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Algorithms
Array signal processing
Beamforming
Cell-free network
Central processing units
Clustering algorithms
Complexity
Computer architecture
CPUs
deep deterministic policy gradient algorithm (DDPG)
Deep learning
deep reinforcement learning (DRL)
distributed distributional deterministic policy gradients algorithm (D4PG)
Electronic devices
Machine learning
Probability density function
Radio equipment
Signal processing
Signal processing algorithms
successive interference cancellation
Transmitters
Uplink
Uplinking
Wireless networks
ISSN:2332-7731, 2332-7731
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:In a cell-free network, a large number of mobile devices are served simultaneously by several base stations (BSs)/access points(APs) using the same time/frequency resources. However, this creates high signal processing demands (e.g., for beamforming) at the transmitters and receivers. In this work, we develop centralized and distributed deep reinforcement learning (DRL)-based methods to optimize beamforming at the uplink of a cell-free network. First, we propose a fully centralized uplink beamforming method (i.e., centralized learning) that uses the Deep Deterministic Policy Gradient algorithm (DDPG) for an offline-trained DRL model. We then enhance this method, in terms of convergence and performance, by using distributed experiences collected from different APs based on the Distributed Distributional Deterministic Policy Gradients algorithm (D4PG) in which the APs represent the distributed agents of the DRL model. To reduce the complexity of signal processing at the central processing unit (CPU), we propose a fully distributed DRL-based uplink beamforming scheme. This scheme divides the beamforming computations among distributed APs. The proposed schemes are then benchmarked against two common linear beamforming schemes, namely, minimum mean square estimation (MMSE) and the simplified conjugate symmetric schemes. The results show that the D4PG scheme with distributed experience achieves the best performance irrespective of the network size. Furthermore, although the proposed distributed beamforming technique reduces the complexity of centralized learning in the DDPG algorithm, it performs better than the DDPG algorithm only for small-scale networks. The performance superiority of the fully centralized DDPG model becomes more evident as the number of APs and/or UEs increases. The codes for all of our DRL implementations are available at https://github.com/RayRedd/Distributed_beamforming_rl .
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2332-7731
2332-7731
DOI:10.1109/TCCN.2022.3165810