Three-Dimensional Trajectory Design for Multi-User MISO UAV Communications: A Deep Reinforcement Learning Approach

In this paper, we investigate a multi-user downlink multiple-input single-output (MISO) unmanned aerial vehicle (UAV) communication system, where a multi-antenna UAV is employed to serve multiple ground terminals. Unlike existing approaches focus only on a simplified two-dimensional scenario, this p...

Full description

Saved in:
Bibliographic Details
Published in:2021 IEEE/CIC International Conference on Communications in China (ICCC) pp. 706 - 711
Main Authors: Wang, Yang, Gao, Zhen
Format: Conference Proceeding
Language:English
Published: IEEE 28.07.2021
Subjects:
3D trajectory design
deep reinforcement learning
Downlink
MISO communication
Multi-antenna UAV
Reinforcement learning
Simulation
Three-dimensional displays
Throughput
UAV communication systems
Urban areas
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract In this paper, we investigate a multi-user downlink multiple-input single-output (MISO) unmanned aerial vehicle (UAV) communication system, where a multi-antenna UAV is employed to serve multiple ground terminals. Unlike existing approaches focus only on a simplified two-dimensional scenario, this paper considers a three-dimensional (3D) urban environment, where the UAV's 3D trajectory is designed to minimize data transmission completion time subject to practical throughput and flight movement constraints. Specifically, we propose a deep reinforcement learning (DRL)-based trajectory design for completion time minimization (DRL- TDCTM), which is developed from a deep deterministic policy gradient algorithm. In particular, to represent the state information of UAV and environment, we set an additional information, i.e., the merged pheromone, as a reference of reward which facilitates the algorithm design. By interacting with the external environment in the corresponding Markov decision process, the proposed algorithm can continuously and adaptively learn how to adjust the UAV's movement strategy. Finally, simulation results show the superiority of the proposed DRL- TDCTM algorithm over the conventional baseline methods.
AbstractList In this paper, we investigate a multi-user downlink multiple-input single-output (MISO) unmanned aerial vehicle (UAV) communication system, where a multi-antenna UAV is employed to serve multiple ground terminals. Unlike existing approaches focus only on a simplified two-dimensional scenario, this paper considers a three-dimensional (3D) urban environment, where the UAV's 3D trajectory is designed to minimize data transmission completion time subject to practical throughput and flight movement constraints. Specifically, we propose a deep reinforcement learning (DRL)-based trajectory design for completion time minimization (DRL- TDCTM), which is developed from a deep deterministic policy gradient algorithm. In particular, to represent the state information of UAV and environment, we set an additional information, i.e., the merged pheromone, as a reference of reward which facilitates the algorithm design. By interacting with the external environment in the corresponding Markov decision process, the proposed algorithm can continuously and adaptively learn how to adjust the UAV's movement strategy. Finally, simulation results show the superiority of the proposed DRL- TDCTM algorithm over the conventional baseline methods.
Author Wang, Yang
Gao, Zhen
Author_xml – sequence: 1
  givenname: Yang
  surname: Wang
  fullname: Wang, Yang
  organization: School of Information and Electronics, Beijing Institute of Technology,Beijing,China,100081
– sequence: 2
  givenname: Zhen
  surname: Gao
  fullname: Gao, Zhen
  email: gaozhen16@bit.edu.cn
  organization: School of Information and Electronics, Beijing Institute of Technology,Beijing,China,100081
BookMark eNotkM1Og0AYRcdEF7b6BCZmXgCcX5i6I1QrCaaJgttmZvhox8BABrro20tiV_dszlncFbr1gweEnimJKSWblyLPc8nSNI0ZYTTeSEUEoTdoRZNECsGVlPcoVKcAEG1dD35yg9cdroL-BTsP4YK3MLmjx-0Q8Oe5m11UT7Bg8b3HdfaD86Hvz95ZPS_m9IqzRYARf4Hzi2Jhac64BB2880ecjWMYtD09oLtWdxM8XneN6ve3Kv-Iyv2uyLMycozwOZLSNmkqgCqhjNIto5YRIJwwYgVtuWVJy4wFSRVYq6whzOjGNJQm3Kik4Wv09N91AHAYg-t1uByuN_A_zaZYwg
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1109/ICCC52777.2021.9580401
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Xplore POP ALL
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
EISBN 1665443855
9781665443852
EndPage 711
ExternalDocumentID 9580401
Genre orig-research
GrantInformation_xml – fundername: NSFC
  grantid: 62071044,61827901
  funderid: 10.13039/501100001809
GroupedDBID 6IE
6IL
CBEJK
RIE
RIL
ID FETCH-LOGICAL-i203t-55cd774e1848b8af21c20e03020c41f3c26f2bce518ecc8cb02badbd1163b86d3
IEDL.DBID RIE
ISICitedReferencesCount 1
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001353334200123&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Thu Jun 29 18:38:11 EDT 2023
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i203t-55cd774e1848b8af21c20e03020c41f3c26f2bce518ecc8cb02badbd1163b86d3
PageCount 6
ParticipantIDs ieee_primary_9580401
PublicationCentury 2000
PublicationDate 2021-July-28
PublicationDateYYYYMMDD 2021-07-28
PublicationDate_xml – month: 07
  year: 2021
  text: 2021-July-28
  day: 28
PublicationDecade 2020
PublicationTitle 2021 IEEE/CIC International Conference on Communications in China (ICCC)
PublicationTitleAbbrev ICCC
PublicationYear 2021
Publisher IEEE
Publisher_xml – name: IEEE
Score 1.8003404
Snippet In this paper, we investigate a multi-user downlink multiple-input single-output (MISO) unmanned aerial vehicle (UAV) communication system, where a...
SourceID ieee
SourceType Publisher
StartPage 706
SubjectTerms 3D trajectory design
deep reinforcement learning
Downlink
MISO communication
Multi-antenna UAV
Reinforcement learning
Simulation
Three-dimensional displays
Throughput
UAV communication systems
Urban areas
Title Three-Dimensional Trajectory Design for Multi-User MISO UAV Communications: A Deep Reinforcement Learning Approach
URI https://ieeexplore.ieee.org/document/9580401
WOSCitedRecordID wos001353334200123&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA61ePCk0opvcvBo2iT7SNZb2VosaC3alt5KHhOph7b0IfjvzaZLpeDFS1jCLoGZLDPfJN83CN1Z7qiKnCMm9kPsAQbR0ibEKGUlVUaJ0K1h9Cx6PTkeZ_0Kut9xYQAgXD6DRvEYzvLt3GyKUlkzS6Tfcx7rHAiRbrlaJemX0azZzfM84UIIj_o4a5Qv73VNCUGjc_y_5U5Q_Zd9h_u7uHKKKjCroeXAGx1IuxDj3wppYB9mPkPN_Ru3wz0M7BNQHBi1ZOh3Fn7pvr_iYWuE91ggqwfc8h_AAr9BkE01oUKIS6XVD9wqZcbraNh5HORPpOyXQKacRmuSJMb6bA48aJNaKseZ4RT8X8ypiZmLDE8d1wYSJr3jpNGUa2W1ZT4n0zK10RmqzuYzOEeYMaVT67TKAGLnUbKjmbCOKZs4zVO4QLXCXpPFVhJjUprq8u_pK3RUuKQoiXJ5jarr5QZu0KH5Wk9Xy9vgxx9eyaM8
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LawIxEA5iC-2pLVr6bg49NprEfWR7k7WiVK20Kt4kz2IPKj4K_fedjYtF6KWXsIRdAjNZZr5Jvm8QejDcUVlzjugAhgAABlHChERLaQSVWsa-W8OoE_d6YjxO-gX0uOPCWGv95TNbyR79Wb6Z601WKqsmoYA9B1jnIAwCTrdsrZz2y2hSbadpGvI4jgH3cVbJX9_rm-LDRvPkfwueovIv_w73d5HlDBXsrISWAzC7JY1Mjn8rpYEh0Hz6qvs3bvibGBhSUOw5tWQIewt32--veFgf4T0eyOoJ1-EDu8Bv1gunal8jxLnW6geu50LjZTRsPg_SFsk7JpApp7U1CUNtIJ-zANuEEtJxpjm18B9zqgPmappHjittQybAdUIrypU0yjDIypSITO0cFWfzmb1AmDGpIuOUTKwNHOBkR5PYOCZN6BSP7CUqZfaaLLaiGJPcVFd_T9-jo9ag25l02r2Xa3ScuScrkHJxg4rr5cbeokP9tZ6ulnfepz8HXKaD
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2021+IEEE%2FCIC+International+Conference+on+Communications+in+China+%28ICCC%29&rft.atitle=Three-Dimensional+Trajectory+Design+for+Multi-User+MISO+UAV+Communications%3A+A+Deep+Reinforcement+Learning+Approach&rft.au=Wang%2C+Yang&rft.au=Gao%2C+Zhen&rft.date=2021-07-28&rft.pub=IEEE&rft.spage=706&rft.epage=711&rft_id=info:doi/10.1109%2FICCC52777.2021.9580401&rft.externalDocID=9580401