Reinforcement Learning Based Network Coding for Drone-Aided Secure Wireless Communications

Active eavesdropper sends jamming signals to raise the transmit power of base stations and steal more information from cellular systems. Network coding resists the active eavesdroppers that cannot obtain all the data flows, but highly relies on the wiretap channel states that are rarely known in wir...

Full description

Saved in:

Bibliographic Details
Published in:	IEEE transactions on communications Vol. 70; no. 9; pp. 5975 - 5988
Main Authors:	Xiao, Liang, Li, Hongyan, Yu, Shi, Zhang, Yi, Wang, Li-Chun, Ma, Shaodan
Format:	Journal Article
Language:	English
Published:	New York IEEE 01.09.2022 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:	Cellular communication Coding Drones Eavesdropping Energy consumption Jamming Learning Network coding Network latency Neural networks Optimization Radio equipment reinforcement learning Relays Resource management secure communications Wireless communications Wireless networks Wiretapping
ISSN:	0090-6778, 1558-0857
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Abstract	Active eavesdropper sends jamming signals to raise the transmit power of base stations and steal more information from cellular systems. Network coding resists the active eavesdroppers that cannot obtain all the data flows, but highly relies on the wiretap channel states that are rarely known in wireless networks. In this paper, we present a reinforcement learning (RL) based random linear network coding scheme for drone-aided cellular systems to address eavesdropping. In this scheme, the network coding policy, including the encoded packet number, the packet and power allocation, is chosen based on the measured jamming power, previous transmission performance and BS channel states. A virtual model generates simulated experiences to update Q-values besides real experiences for faster policy optimization. We also propose a deep RL version and design a hierarchical architecture to further accelerate the policy exploration and improve the anti-eavesdropping performance, in terms of the intercept probability, the latency, the outage probability and the energy consumption. We analyze the computational complexity, drone deployment, secure coverage area and the performance bound of the proposed schemes, which are verified via simulation results.
AbstractList	Active eavesdropper sends jamming signals to raise the transmit power of base stations and steal more information from cellular systems. Network coding resists the active eavesdroppers that cannot obtain all the data flows, but highly relies on the wiretap channel states that are rarely known in wireless networks. In this paper, we present a reinforcement learning (RL) based random linear network coding scheme for drone-aided cellular systems to address eavesdropping. In this scheme, the network coding policy, including the encoded packet number, the packet and power allocation, is chosen based on the measured jamming power, previous transmission performance and BS channel states. A virtual model generates simulated experiences to update Q-values besides real experiences for faster policy optimization. We also propose a deep RL version and design a hierarchical architecture to further accelerate the policy exploration and improve the anti-eavesdropping performance, in terms of the intercept probability, the latency, the outage probability and the energy consumption. We analyze the computational complexity, drone deployment, secure coverage area and the performance bound of the proposed schemes, which are verified via simulation results.
Author	Xiao, Liang Ma, Shaodan Yu, Shi Zhang, Yi Li, Hongyan Wang, Li-Chun
Author_xml	– sequence: 1 givenname: Liang orcidid: 0000-0003-2402-611X surname: Xiao fullname: Xiao, Liang organization: Department of Information and Communication Engineering, Xiamen University, Xiamen, China – sequence: 2 givenname: Hongyan orcidid: 0000-0002-0697-4709 surname: Li fullname: Li, Hongyan organization: Department of Information and Communication Engineering, Xiamen University, Xiamen, China – sequence: 3 givenname: Shi orcidid: 0000-0002-1946-621X surname: Yu fullname: Yu, Shi organization: Department of Information and Communication Engineering, Xiamen University, Xiamen, China – sequence: 4 givenname: Yi orcidid: 0000-0001-8991-1983 surname: Zhang fullname: Zhang, Yi email: yizhang@xmu.edu.cn organization: Department of Information and Communication Engineering, Xiamen University, Xiamen, China – sequence: 5 givenname: Li-Chun orcidid: 0000-0002-7883-6217 surname: Wang fullname: Wang, Li-Chun email: lichun@cc.nctu.edu.tw organization: Department of Electrical and Computer Engineering, National Chiao Tung University, Hsinchu, Taiwan – sequence: 6 givenname: Shaodan orcidid: 0000-0001-5521-3650 surname: Ma fullname: Ma, Shaodan email: shaodanma@um.edu.mo organization: State Key Laboratory of Internet of Things for Smart City and the Department of Electrical and Computer Engineering, University of Macau, Macao, SAR, China
BookMark	eNp9kM1LAzEQxYNUsK3-A3pZ8Lx1kk02ybHWT6gWtCJ4WbK7U0ltszXZIv73ph948OBpYOa9ebxfj3Rc45CQUwoDSkFfTEeTh4cBA8YGGdUcJD8gXSqESkEJ2SFdAA1pLqU6Ir0Q5gDAIcu65O0JrZs1vsIlujYZo_HOuvfk0gSsk0dsvxr_kYyaerOMuuTKx-R0aOt4fsZq7TF5tR4XGEKULZdrZyvT2saFY3I4M4uAJ_vZJy8319PRXTqe3N6PhuO0Ylq0qYBclFwDmhlDpCXPKy4xZ7xUqqYoeJ0LVAIlCDQlUmQllcpgpmMHUemsT853f1e--VxjaIt5s_YuRhZMUq4FVwyiiu1UlW9C8DgrVt4ujf8uKBQbhsWWYbFhWOwZRpP6Y6psu23XemMX_1vPdlaLiL9ZWkXskmU_mxSBfA
CODEN	IECMBT
CitedBy_id	crossref_primary_10_1109_TCOMM_2023_3341856 crossref_primary_10_1109_TMC_2024_3505206 crossref_primary_10_1109_TWC_2024_3454073 crossref_primary_10_1109_TMC_2024_3502685 crossref_primary_10_1109_ACCESS_2024_3363890 crossref_primary_10_3390_jmse12060998 crossref_primary_10_1109_COMST_2022_3224279 crossref_primary_10_1002_nem_2297 crossref_primary_10_1109_JIOT_2024_3361447 crossref_primary_10_2139_ssrn_5053242 crossref_primary_10_1109_JIOT_2025_3543199 crossref_primary_10_1145_3703625 crossref_primary_10_1109_LWC_2025_3542374 crossref_primary_10_1109_TIV_2023_3274671
Cites_doi	10.1109/TWC.2017.2764891 10.1109/TVT.2020.2967026 10.1016/j.patcog.2006.04.041 10.1109/GLOCOM.2017.8255054 10.1109/TWC.2018.2838592 10.1109/GLOBECOM46510.2021.9685317 10.1109/INFOCOM.2015.7218560 10.1145/122344.122377 10.1109/JSAC.2013.130917 10.1109/JIOT.2018.2875065 10.1109/TWC.2020.3022399 10.1109/TWC.2016.2617328 10.1109/TMM.2007.907460 10.1109/TIT.2017.2710192 10.1109/TIT.2021.3116962 10.1109/TCOMM.2019.2930247 10.1109/TIFS.2019.2954748 10.1109/TMM.2016.2545403 10.1109/TIT.2006.881746 10.1109/TWC.2020.3024860 10.1109/TWC.2011.110711.102126 10.1109/TIFS.2019.2912074 10.1109/TWC.2014.2314713 10.1109/TWC.2017.2717822 10.1109/tnn.1998.712192 10.1109/TVT.2021.3077662 10.1109/TWC.2012.120412.112266 10.1109/TCOMM.2016.2645679 10.1109/TCOMM.2018.2792014 10.1109/TCOMM.2019.2911578 10.1109/TWC.2020.3000303 10.1109/TCOMM.2019.2892474 10.1109/TIFS.2021.3103062 10.1109/TWC.2019.2906177 10.1109/TNN.2003.809401 10.1109/TCOMM.2020.3043776 10.1109/TWC.2020.3034457 10.1109/TWC.2019.2892461 10.1109/JSAC.2021.3088672 10.1109/TVT.2017.2782726
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022
DBID	97E RIA RIE AAYXX CITATION 7SP 8FD L7M
DOI	10.1109/TCOMM.2022.3194074
DatabaseName	IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Electronics & Communications Abstracts Technology Research Database Advanced Technologies Database with Aerospace
DatabaseTitle	CrossRef Technology Research Database Advanced Technologies Database with Aerospace Electronics & Communications Abstracts
DatabaseTitleList	Technology Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	1558-0857
EndPage	5988
ExternalDocumentID	10_1109_TCOMM_2022_3194074 9840372
Genre	orig-research
GrantInformation_xml	– fundername: Science and Technology Development Fund, Macau SAR grantid: 0036/2019/A1; SKL-IOTSC(UM)-2021-2023) – fundername: National Natural Science Foundation of China grantid: U21A20444; 61971366 funderid: 10.13039/501100001809 – fundername: Research Committee of University of Macau grantid: MYRG2020-00095-FST funderid: 10.13039/501100004733
GroupedDBID	-~X .DC 0R~ 29I 3EH 4.4 5GY 5VS 6IK 85S 97E AAJGR AARMG AASAJ AAWTH ABAZT ABFSI ABQJQ ABVLG ACGFO ACGFS ACIWK ACKIV ACNCT AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 E.L EBS EJD HZ~ H~9 IAAWW IBMZZ ICLAB IES IFIPE IFJZH IPLJI JAVBF LAI M43 MS~ O9- OCL P2P RIA RIE RNS TAE TN5 VH1 ZCA ZCG AAYXX CITATION 7SP 8FD L7M
ID	FETCH-LOGICAL-c295t-5065b490eaf2ee1b46c47e624b88d1e54d65e85e705eabe1e2b178ae390045c93
IEDL.DBID	RIE
ISICitedReferencesCount	19
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000854601700024&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0090-6778
IngestDate	Mon Jun 30 10:12:50 EDT 2025 Tue Nov 18 21:27:19 EST 2025 Sat Nov 29 04:08:24 EST 2025 Wed Aug 27 02:15:01 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	9
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c295t-5065b490eaf2ee1b46c47e624b88d1e54d65e85e705eabe1e2b178ae390045c93
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0001-8991-1983 0000-0001-5521-3650 0000-0003-2402-611X 0000-0002-0697-4709 0000-0002-7883-6217 0000-0002-1946-621X
PQID	2714954820
PQPubID	85472
PageCount	14
ParticipantIDs	proquest_journals_2714954820 crossref_primary_10_1109_TCOMM_2022_3194074 ieee_primary_9840372 crossref_citationtrail_10_1109_TCOMM_2022_3194074
PublicationCentury	2000
PublicationDate	2022-09-01
PublicationDateYYYYMMDD	2022-09-01
PublicationDate_xml	– month: 09 year: 2022 text: 2022-09-01 day: 01
PublicationDecade	2020
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	IEEE transactions on communications
PublicationTitleAbbrev	TCOMM
PublicationYear	2022
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 ref35 ref12 ref34 ref15 ref37 ref14 ref36 ref31 ref30 ref11 ref33 ref10 ref32 ref2 ref1 ref17 ref39 ref38 Jin (ref16) ref24 ref23 ref26 ref25 ref20 ref42 ref41 ref22 ref21 ref43 Brafman (ref18) 2002; 3 ref28 ref27 ref29 ref8 Kulkarni (ref19); 29 ref7 ref9 ref4 ref3 ref6 ref5 ref40
References_xml	– ident: ref15 doi: 10.1109/TWC.2017.2764891 – ident: ref34 doi: 10.1109/TVT.2020.2967026 – ident: ref37 doi: 10.1016/j.patcog.2006.04.041 – ident: ref39 doi: 10.1109/GLOCOM.2017.8255054 – ident: ref6 doi: 10.1109/TWC.2018.2838592 – ident: ref26 doi: 10.1109/GLOBECOM46510.2021.9685317 – ident: ref17 doi: 10.1109/INFOCOM.2015.7218560 – ident: ref35 doi: 10.1145/122344.122377 – ident: ref2 doi: 10.1109/JSAC.2013.130917 – start-page: 4868 volume-title: Proc. Conf. Adv. Neural Inf. Process. Syst. (NIPS) ident: ref16 article-title: Is Q-learning provably efficient? – volume: 3 start-page: 213 year: 2002 ident: ref18 article-title: R-MAX—A general polynomial time algorithm for near-optimal reinforcement learning publication-title: J. Mach. Learn. Res. – ident: ref22 doi: 10.1109/JIOT.2018.2875065 – ident: ref14 doi: 10.1109/TWC.2020.3022399 – ident: ref5 doi: 10.1109/TWC.2016.2617328 – ident: ref30 doi: 10.1109/TMM.2007.907460 – ident: ref42 doi: 10.1109/TIT.2017.2710192 – ident: ref24 doi: 10.1109/TIT.2021.3116962 – ident: ref27 doi: 10.1109/TCOMM.2019.2930247 – ident: ref4 doi: 10.1109/TIFS.2019.2954748 – ident: ref7 doi: 10.1109/TMM.2016.2545403 – ident: ref8 doi: 10.1109/TIT.2006.881746 – ident: ref28 doi: 10.1109/TWC.2020.3024860 – ident: ref41 doi: 10.1109/TWC.2011.110711.102126 – ident: ref1 doi: 10.1109/TIFS.2019.2912074 – ident: ref29 doi: 10.1109/TWC.2014.2314713 – ident: ref33 doi: 10.1109/TWC.2017.2717822 – ident: ref40 doi: 10.1109/tnn.1998.712192 – ident: ref12 doi: 10.1109/TVT.2021.3077662 – ident: ref31 doi: 10.1109/TWC.2012.120412.112266 – ident: ref32 doi: 10.1109/TCOMM.2016.2645679 – ident: ref20 doi: 10.1109/TCOMM.2018.2792014 – ident: ref9 doi: 10.1109/TCOMM.2019.2911578 – ident: ref10 doi: 10.1109/TWC.2020.3000303 – ident: ref23 doi: 10.1109/TCOMM.2019.2892474 – ident: ref25 doi: 10.1109/TIFS.2021.3103062 – ident: ref3 doi: 10.1109/TWC.2019.2906177 – ident: ref43 doi: 10.1109/TNN.2003.809401 – ident: ref38 doi: 10.1109/TCOMM.2020.3043776 – ident: ref13 doi: 10.1109/TWC.2020.3034457 – volume: 29 start-page: 3675 volume-title: Proc. Conf. Adv. Neural Inf. Process. Syst. (NIPS) ident: ref19 article-title: Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation – ident: ref21 doi: 10.1109/TWC.2019.2892461 – ident: ref11 doi: 10.1109/JSAC.2021.3088672 – ident: ref36 doi: 10.1109/TVT.2017.2782726
SSID	ssj0004033
Score	2.4872036
Snippet	Active eavesdropper sends jamming signals to raise the transmit power of base stations and steal more information from cellular systems. Network coding resists...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	5975
SubjectTerms	Cellular communication Coding Drones Eavesdropping Energy consumption Jamming Learning Network coding Network latency Neural networks Optimization Radio equipment reinforcement learning Relays Resource management secure communications Wireless communications Wireless networks Wiretapping
Title	Reinforcement Learning Based Network Coding for Drone-Aided Secure Wireless Communications
URI	https://ieeexplore.ieee.org/document/9840372 https://www.proquest.com/docview/2714954820
Volume	70
WOSCitedRecordID	wos000854601700024&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-0857 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0004033 issn: 0090-6778 databaseCode: RIE dateStart: 19720101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NS8MwFA9zeNCDX1OcTsnBm8a1aZqP45wOD26KTBheSpO-ymBssg__fpO0mw5F8FZIUkJ-7fvKe7-H0AUwGbjCPJJDyokjoCOSC0O4oTTnAMAyT5n_IHo9ORiopwq6WtXC2GGffAbX7tHf5WcTs3Chsqay3kgkrMDdEIIXtVpfNZBBVDJOunR2IZcFMoFq9tuP3a51BSm1HqqyHgxbU0K-q8oPUez1S2f3fzvbQzulHYlbBfD7qALjA7T9jV2whl6fwdOiGh8BxCWT6hu-sYorw70i_xu3J057YTsP304nYyCtYWaHfRwesEuOHVlhiNcKSWaH6KVz12_fk7KVAjFUxXMSW0tDMxVAmlOAUDNumABOmZYyCyFmGY9BxiCCGFINIVAdCplCpJzNZ1R0hKpju4djhC2elOd5FjCtWJzKVGvBmM6YtW200aqOwuXZJqbkGXftLkaJ9zcClXg8EodHUuJRR5erNe8Fy8afs2sOgdXM8vDrqLGEMCl_xFlChXMBmbVzTn5fdYq23LuLtLEGqs6nCzhDm-ZjPpxNz_039gmjkc5T
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NS8MwFH8MFdSD3-J0ag7etK5N0yY56lQUtyoyYXgpTfoqA9lkm_79Jmk3FUXwVkhCQ37t-8p7vwdwhEz4tjDPKzCLPUtA54mYay_WlBYxIrLcUea3eZKIXk_e1-BkVgtjhl3yGZ7aR3eXnw_1mw2VNaXxRkJuBO58xBj1y2qtzypIP6w4J21COxfTEhlfNrutu07HOIOUGh9VGh-GfVNDrq_KD2HsNMzV6v_2tgYrlSVJzkro16GGgw1Y_sIvuAlPD-iIUbWLAZKKS_WZnBvVlZOkzAAnraHVX8TMIxej4QC9s35uhl0kHolNj30x4pB8KyUZb8Hj1WW3de1VzRQ8TWU08SJjaygmfcwKihgoFmvGMaZMCZEHGLE8jlBEyP0IM4UBUhVwkWEordWnZbgNcwOzhx0gBlEaF0XuMyVZlIlMKc6YypmxbpRWsg7B9GxTXTGN24YXL6nzOHyZOjxSi0da4VGH49ma15Jn48_ZmxaB2czq8OvQmEKYVr_iOKXcOoHMWDq7v686hMXrbqedtm-S2z1Ysu8pk8gaMDcZveE-LOj3SX88OnDf2weIRNGa
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Reinforcement+Learning+Based+Network+Coding+for+Drone-Aided+Secure+Wireless+Communications&rft.jtitle=IEEE+transactions+on+communications&rft.au=Liang%2C+Xiao&rft.au=Li%2C+Hongyan&rft.au=Shi%2C+Yu&rft.au=Zhang%2C+Yi&rft.date=2022-09-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.issn=0090-6778&rft.eissn=1558-0857&rft.volume=70&rft.issue=9&rft.spage=5975&rft_id=info:doi/10.1109%2FTCOMM.2022.3194074&rft.externalDBID=NO_FULL_TEXT
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0090-6778&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0090-6778&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0090-6778&client=summon