Joint Multi-objective Optimization for Radio Access Network Slicing Using Multi-agent Deep Reinforcement Learning

Radio access network (RAN) slices can provide various customized services for next-generation wireless networks. Thus, multiple performance metrics of different types of RAN slices need to be jointly optimized. However, existing efforts in multi-objective optimization problem (MOOP) for RAN slicing...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on vehicular technology Vol. 72; no. 9; pp. 1 - 16
Main Authors: Zhou, Guorong, Zhao, Liqiang, Zheng, Gan, Xie, Zhijie, Song, Shenghui, Chen, Kwang-Cheng
Format: Journal Article
Language:English
Published: New York IEEE 01.09.2023
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
ISSN:0018-9545, 1939-9359
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Radio access network (RAN) slices can provide various customized services for next-generation wireless networks. Thus, multiple performance metrics of different types of RAN slices need to be jointly optimized. However, existing efforts in multi-objective optimization problem (MOOP) for RAN slicing are only in the scalar form, which is difficult to achieve simultaneous optimization. In this paper, we consider a non-scalar MOOP for RAN slicing with three types of slices, i.e. , the high-bandwidth slice, the low-delay slice, and the wide-coverage slice over the same underlying physical network. We jointly optimize the throughput, the transmission delay, and the coverage area by user-oriented dynamic virtual base stations (vBSs)' deployment, and sub-channel and power allocation. An improved multi-agent deep deterministic policy gradient (IMADDPG) algorithm, having the characteristics of centralized training and distributed execution, is proposed to solve the above non-deterministic polynomial-time hard (NP-hard) problem. The rank voting method is introduced in the inference process to obtain near-Pareto optimal solutions. Simulation results verify that the proposed scheme can ensure better performance than the traditional scalar utility method and other benchmark algorithms. The proposed scheme has the advantage of flexibly approaching any point of the Pareto boundary, while the traditional scalar method only subjectively approaches one of the Pareto optimal solutions. Furthermore, our proposal strikes a compelling tradeoff among three types of RAN slices due to the non-dominance between Pareto optimal solutions.
AbstractList Radio access network (RAN) slices can provide various customized services for next-generation wireless networks. Thus, multiple performance metrics of different types of RAN slices need to be jointly optimized. However, existing efforts in multi-objective optimization problem (MOOP) for RAN slicing are only in the scalar form, which is difficult to achieve simultaneous optimization. In this paper, we consider a non-scalar MOOP for RAN slicing with three types of slices, i.e., the high-bandwidth slice, the low-delay slice, and the wide-coverage slice over the same underlying physical network. We jointly optimize the throughput, the transmission delay, and the coverage area by user-oriented dynamic virtual base stations (vBSs)’ deployment, and sub-channel and power allocation. An improved multi-agent deep deterministic policy gradient (IMADDPG) algorithm, having the characteristics of centralized training and distributed execution, is proposed to solve the above non-deterministic polynomial-time hard (NP-hard) problem. The rank voting method is introduced in the inference process to obtain near-Pareto optimal solutions. Simulation results verify that the proposed scheme can ensure better performance than the traditional scalar utility method and other benchmark algorithms. The proposed scheme has the advantage of flexibly approaching any point of the Pareto boundary, while the traditional scalar method only subjectively approaches one of the Pareto optimal solutions. Furthermore, our proposal strikes a compelling tradeoff among three types of RAN slices due to the non-dominance between Pareto optimal solutions.
Author Zheng, Gan
Song, Shenghui
Xie, Zhijie
Zhao, Liqiang
Chen, Kwang-Cheng
Zhou, Guorong
Author_xml – sequence: 1
  givenname: Guorong
  orcidid: 0000-0001-9430-1397
  surname: Zhou
  fullname: Zhou, Guorong
  organization: State Key Laboratory of Integrated Service Networks, Xidian University, Xi'an, China
– sequence: 2
  givenname: Liqiang
  orcidid: 0000-0002-3374-6066
  surname: Zhao
  fullname: Zhao, Liqiang
  organization: State Key Laboratory of Integrated Service Networks, Xidian University, Xi'an, China
– sequence: 3
  givenname: Gan
  orcidid: 0000-0001-8457-6477
  surname: Zheng
  fullname: Zheng, Gan
  organization: School of Engineering, University of Warwick, Coventry, U.K
– sequence: 4
  givenname: Zhijie
  surname: Xie
  fullname: Xie, Zhijie
  organization: Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong
– sequence: 5
  givenname: Shenghui
  orcidid: 0000-0001-6316-8415
  surname: Song
  fullname: Song, Shenghui
  organization: Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong
– sequence: 6
  givenname: Kwang-Cheng
  orcidid: 0000-0002-1024-6106
  surname: Chen
  fullname: Chen, Kwang-Cheng
  organization: Department of Electrical Engineering, University of South Florida, Tampa, FL, USA
BookMark eNp9kE1PAjEURRujiYjuXbho4nqw7bSd6ZL4bVASBLeTtrwhRZhiWzT66x2EhXHh5jWvuee-5Byh_cY3gNApJT1KiboYv4x7jLC8lzNZyoLuoQ5VucpULtQ-6hBCy0wJLg7RUYzzduVc0Q56e_CuSfhxvUgu82YONrl3wMNVckv3pZPzDa59wCM9dR73rYUY8ROkDx9e8fPCWdfM8CRu5rZDz6DtuwJY4RG4pmUtLDdfA9ChaXPH6KDWiwgnu7eLJjfX48u7bDC8vb_sDzLLFEuZNsQUhheSUas4sGkpSWFyQZQhYIUoeF6XjJqpoLVQNTeac2ZkqaSYAqlZ3kXn295V8G9riKma-3Vo2pMVK6XkihBRtimyTdngYwxQV6vgljp8VpRUG7FVK7baiK12YltE_kGsSz-mUtBu8R94tgUdAPy6Q4kkVOTfOLyIaQ
CODEN ITVTAB
CitedBy_id crossref_primary_10_1109_COMST_2023_3338153
crossref_primary_10_1109_LWC_2024_3365161
crossref_primary_10_1109_TPDS_2023_3310013
crossref_primary_10_1109_ACCESS_2025_3538546
crossref_primary_10_1109_TNSM_2024_3476480
crossref_primary_10_1109_TSC_2025_3586152
crossref_primary_10_1109_TVT_2024_3431878
crossref_primary_10_1007_s11082_023_05796_4
crossref_primary_10_1109_ACCESS_2023_3296851
crossref_primary_10_1109_TGCN_2024_3397459
crossref_primary_10_54392_irjmt24324
crossref_primary_10_1109_JIOT_2023_3319130
crossref_primary_10_1109_JIOT_2024_3416157
crossref_primary_10_1109_TCOMM_2023_3335414
Cites_doi 10.1109/TWC.2017.2789294
10.1109/TVT.2022.3202689
10.1109/TWC.2021.3123500
10.1109/TNSE.2018.2876918
10.1109/SPAWC.2017.8227791
10.1109/JIOT.2018.2888543
10.1609/aaai.v32i1.11694
10.1109/MSP.2014.2330661
10.1109/JIOT.2021.3068518
10.1109/COMST.2016.2610578
10.1109/TWC.2017.2696000
10.1109/ACCESS.2019.2902432
10.1109/TCOMM.2022.3211083
10.1109/TVT.2019.2952216
10.1109/JSAC.2019.2959185
10.1109/MVT.2020.3015184
10.1109/TVT.2019.2896586
10.1109/TWC.2017.2725836
10.1109/TMC.2019.2930059
10.1162/106365600568158
10.1109/TWC.2020.2965927
10.1109/TVT.2021.3095901
10.1109/JSAC.2019.2933893
10.1109/JIOT.2021.3111644
10.1109/TNSM.2019.2899609
10.1109/JSAC.2020.2986869
10.1109/TVT.2007.912960
10.1109/COMST.2018.2815638
10.1109/ACCESS.2018.2822398
10.1109/ACCESS.2020.3036416
10.1109/TVT.2017.2738024
10.1109/JIOT.2021.3089823
10.1109/TKDE.2012.73
10.1109/MCOM.2014.6766097
10.1109/JIOT.2020.2997342
10.1109/MCOM.2017.1600951
10.1109/COMST.2020.2965856
10.1109/ACCESS.2018.2881964
10.1109/TVT.2020.2972999
10.1038/nature24270
10.1109/MVT.2021.3085511
10.1016/j.neucom.2017.02.096
10.1109/OJVT.2021.3095467
10.1109/TCOMM.2021.3090423
10.1109/PESGM.2017.8274155
10.1109/TNSM.2019.2945254
10.1109/ACCESS.2019.2909670
ContentType Journal Article
Copyright Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023
Copyright_xml – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023
DBID 97E
RIA
RIE
AAYXX
CITATION
7SP
8FD
FR3
KR7
L7M
DOI 10.1109/TVT.2023.3268671
DatabaseName IEEE All-Society Periodicals Package (ASPP) 2005–Present
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE Electronic Library (IEL)
CrossRef
Electronics & Communications Abstracts
Technology Research Database
Engineering Research Database
Civil Engineering Abstracts
Advanced Technologies Database with Aerospace
DatabaseTitle CrossRef
Civil Engineering Abstracts
Engineering Research Database
Technology Research Database
Advanced Technologies Database with Aerospace
Electronics & Communications Abstracts
DatabaseTitleList Civil Engineering Abstracts

Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
EISSN 1939-9359
EndPage 16
ExternalDocumentID 10_1109_TVT_2023_3268671
10106015
Genre orig-research
GrantInformation_xml – fundername: Key-Area Research and Development Program of Guangdong Province
  grantid: 2020B0101120003
– fundername: National Natural Science Foundation of China; NSFC
  grantid: 62071352
  funderid: 10.13039/501100001809
– fundername: National Key R&D Program of China
  grantid: 2020YFB1807700; 2020YFB1806404
– fundername: Key Research and Development Projects of Shaanxi Province; Key Research and Development Program of Shaanxi
  grantid: 2022KWZ-
  funderid: 10.13039/501100015401
– fundername: The Hong Kong University of Science and Technology (HKUST) Startup Fund
  grantid: R9249
– fundername: Higher Education Discipline Innovation Project; 111 Project
  grantid: B08038
  funderid: 10.13039/501100013314
GroupedDBID -~X
.DC
0R~
29I
4.4
5GY
6IK
97E
AAIKC
AAJGR
AAMNW
AARMG
AASAJ
AAWTH
ABAZT
ABQJQ
ABVLG
ACGFO
ACGFS
ACIWK
ACNCT
AENEX
AGQYO
AHBIQ
AKJIK
AKQYR
ALMA_UNASSIGNED_HOLDINGS
ASUFR
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CS3
DU5
EBS
HZ~
IFIPE
IPLJI
JAVBF
LAI
M43
MS~
O9-
OCL
P2P
RIA
RIE
RNS
RXW
TAE
TN5
3EH
5VS
AAYXX
AETIX
AGSQL
AI.
AIBXA
ALLEH
CITATION
EJD
H~9
IAAWW
IBMZZ
ICLAB
IFJZH
VH1
7SP
8FD
FR3
KR7
L7M
ID FETCH-LOGICAL-c292t-ab0b7b47621c94e2d8607b3509b0ec55743f821bd51f59f4ba442b68965de0f23
IEDL.DBID RIE
ISICitedReferencesCount 15
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001103676800058&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 0018-9545
IngestDate Mon Jun 30 10:15:01 EDT 2025
Tue Nov 18 22:20:45 EST 2025
Sat Nov 29 02:59:08 EST 2025
Wed Aug 27 02:14:16 EDT 2025
IsPeerReviewed true
IsScholarly true
Issue 9
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
https://doi.org/10.15223/policy-029
https://doi.org/10.15223/policy-037
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c292t-ab0b7b47621c94e2d8607b3509b0ec55743f821bd51f59f4ba442b68965de0f23
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0000-0001-9430-1397
0000-0002-1024-6106
0000-0001-8457-6477
0000-0001-6316-8415
0000-0002-3374-6066
0000-0002-2003-0472
PQID 2866490058
PQPubID 85454
PageCount 16
ParticipantIDs crossref_primary_10_1109_TVT_2023_3268671
crossref_citationtrail_10_1109_TVT_2023_3268671
proquest_journals_2866490058
ieee_primary_10106015
PublicationCentury 2000
PublicationDate 2023-09-01
PublicationDateYYYYMMDD 2023-09-01
PublicationDate_xml – month: 09
  year: 2023
  text: 2023-09-01
  day: 01
PublicationDecade 2020
PublicationPlace New York
PublicationPlace_xml – name: New York
PublicationTitle IEEE transactions on vehicular technology
PublicationTitleAbbrev TVT
PublicationYear 2023
Publisher IEEE
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml – name: IEEE
– name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References ref13
ref12
ref15
ref14
ref11
ref10
ref17
ref16
ref19
ref18
ref46
ref45
ref48
ref47
ref42
ref41
ref44
ref43
ref49
ref8
ref7
ref9
ref4
ref3
ref6
ref5
ref35
ref34
ref37
ref36
ref31
ref30
ref33
ref2
ref1
ref39
ref38
ref24
ref23
ref26
ref25
ref20
ref22
ref21
ref28
ref27
ref29
bertsekas (ref40) 1992
lowe (ref32) 0
References_xml – ident: ref5
  doi: 10.1109/TWC.2017.2789294
– ident: ref49
  doi: 10.1109/TVT.2022.3202689
– ident: ref42
  doi: 10.1109/TWC.2021.3123500
– ident: ref23
  doi: 10.1109/TNSE.2018.2876918
– start-page: 6379
  year: 0
  ident: ref32
  article-title: Multi-agent actor-critic for mixed cooperative-competitive environments
  publication-title: Proc Adv Neural Inf Process Syst
– ident: ref7
  doi: 10.1109/SPAWC.2017.8227791
– year: 1992
  ident: ref40
  publication-title: Data Networks
– ident: ref19
  doi: 10.1109/JIOT.2018.2888543
– ident: ref48
  doi: 10.1609/aaai.v32i1.11694
– ident: ref21
  doi: 10.1109/MSP.2014.2330661
– ident: ref20
  doi: 10.1109/JIOT.2021.3068518
– ident: ref15
  doi: 10.1109/COMST.2016.2610578
– ident: ref41
  doi: 10.1109/TWC.2017.2696000
– ident: ref30
  doi: 10.1109/ACCESS.2019.2902432
– ident: ref34
  doi: 10.1109/TCOMM.2022.3211083
– ident: ref38
  doi: 10.1109/TVT.2019.2952216
– ident: ref9
  doi: 10.1109/JSAC.2019.2959185
– ident: ref13
  doi: 10.1109/MVT.2020.3015184
– ident: ref27
  doi: 10.1109/TVT.2019.2896586
– ident: ref25
  doi: 10.1109/TWC.2017.2725836
– ident: ref22
  doi: 10.1109/TMC.2019.2930059
– ident: ref31
  doi: 10.1162/106365600568158
– ident: ref4
  doi: 10.1109/TWC.2020.2965927
– ident: ref3
  doi: 10.1109/TVT.2021.3095901
– ident: ref39
  doi: 10.1109/JSAC.2019.2933893
– ident: ref2
  doi: 10.1109/JIOT.2021.3111644
– ident: ref24
  doi: 10.1109/TNSM.2019.2899609
– ident: ref33
  doi: 10.1109/JSAC.2020.2986869
– ident: ref29
  doi: 10.1109/TVT.2007.912960
– ident: ref1
  doi: 10.1109/COMST.2018.2815638
– ident: ref17
  doi: 10.1109/ACCESS.2018.2822398
– ident: ref35
  doi: 10.1109/ACCESS.2020.3036416
– ident: ref8
  doi: 10.1109/TVT.2017.2738024
– ident: ref36
  doi: 10.1109/JIOT.2021.3089823
– ident: ref28
  doi: 10.1109/TKDE.2012.73
– ident: ref26
  doi: 10.1109/MCOM.2014.6766097
– ident: ref18
  doi: 10.1109/JIOT.2020.2997342
– ident: ref37
  doi: 10.1109/MCOM.2017.1600951
– ident: ref14
  doi: 10.1109/COMST.2020.2965856
– ident: ref43
  doi: 10.1109/ACCESS.2018.2881964
– ident: ref6
  doi: 10.1109/TVT.2020.2972999
– ident: ref44
  doi: 10.1038/nature24270
– ident: ref10
  doi: 10.1109/MVT.2021.3085511
– ident: ref46
  doi: 10.1016/j.neucom.2017.02.096
– ident: ref11
  doi: 10.1109/OJVT.2021.3095467
– ident: ref47
  doi: 10.1109/TCOMM.2021.3090423
– ident: ref12
  doi: 10.1109/PESGM.2017.8274155
– ident: ref16
  doi: 10.1109/TNSM.2019.2945254
– ident: ref45
  doi: 10.1109/ACCESS.2019.2909670
SSID ssj0014491
Score 2.487317
Snippet Radio access network (RAN) slices can provide various customized services for next-generation wireless networks. Thus, multiple performance metrics of...
SourceID proquest
crossref
ieee
SourceType Aggregation Database
Enrichment Source
Index Database
Publisher
StartPage 1
SubjectTerms Algorithms
Delays
Heuristic algorithms
Hierarchies
multi-agent deep reinforcement learning
multi-objective optimization
Multiagent systems
Multiple objective analysis
Network slicing
non-scalarization
Optimization
Pareto optimization
Pareto optimum
Performance measurement
Polynomials
Radio access network slicing
rank voting method
Resource management
Throughput
Wireless networks
Title Joint Multi-objective Optimization for Radio Access Network Slicing Using Multi-agent Deep Reinforcement Learning
URI https://ieeexplore.ieee.org/document/10106015
https://www.proquest.com/docview/2866490058
Volume 72
WOSCitedRecordID wos001103676800058&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVIEE
  databaseName: IEEE Electronic Library (IEL)
  customDbUrl:
  eissn: 1939-9359
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0014491
  issn: 0018-9545
  databaseCode: RIE
  dateStart: 19670101
  isFulltext: true
  titleUrlDefault: https://ieeexplore.ieee.org/
  providerName: IEEE
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NT8IwFG-EeNCDnxhRND148TDYSteuR6ISYwwaRMJtWbc3gsENYfj323aFYIwm7rQlbbPs9X2s773fD6ErX_tM4K4jAsIdzW3ryIS1HRJHylvIlEhD5zN85L1eMBqJZ9usbnphAMAUn0FT35pcfpLHS31UpjTc0_AhfgVVOGdls9Y6ZUCppcfzlAaruGCVk3RFazAcNDVNeFPFKhrP7ZsPMqQqPyyxcS_d_X--2AHas3Ek7pSCP0RbkB2h3Q10wWP08ZBPsgKbFlsnl2-lacNPyki82-5LrEJW3I-SSY47hjkR98qycPwy1Sn3MTYlBXaNSLdh4VuAGe6DQVyNzeEitiCt4xp67d4Nbu4dy7DgxESQwomkK7mkyiB6saBAkoC5XLZVECFdiH1fhRdpQDyZ-F7qi5TKiFIiWSCYn4CbkvYJqmZ5BqcIM7UZfCaU_aTqAhJ46hE4S9pMcgpeHbVW3zyMLfy4ZsGYhuY3xBWhklKopRRaKdXR9XrGrITe-GNsTUtlY1wpkDpqrOQaWuVchCRgjApNqHj2y7RztKNXL2vJGqhazJdwgbbjz2KymF-affcFbpnUQw
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LT8MwDLZ4ScCB5xCDATlw4dDRZknaHBEP8RgDwUDcqqZ1p6GxDjb4_SRphoYQSPTUSklbxYntxPb3AexzYzMx9D0Z0dAz3LaeykTDo2mirYXKqbJ0Po_NsNWKnp7krStWt7UwiGiTz7Bubm0sPyvSd3NUpld4YOBD-DTMcsaoX5ZrfQUNGHMEeYFew9ozGEclfXnYfmzXDVF4XXsrBtHtmxWytCo_dLE1MGfL__y1FVhyniQ5KkW_ClPYX4PFCXzBdXi9LLr9EbFFtl6hnkvlRm60mnhx9ZdEO63kLsm6BTmy3ImkVSaGk_ueCbp3iE0qcO9ITCEWOUEckDu0mKupPV4kDqa1U4GHs9P28bnnOBa8lEo68hLlq1AxrRKDVDKkWST8UDW0G6F8TDnXDkYe0UBlPMi5zJlK9MArEUnBM_Rz2tiAmX7Rx00gQk8HLqTWoExfSKNAP2IosoZQIcOgCofjMY9TB0BueDB6sd2I-DLWUoqNlGInpSocfPUYlOAbf7StGKlMtCsFUoXaWK6xW57DmEZCMGkoFbd-6bYH8-ft62bcvGhdbcOC-VKZWVaDmdHbO-7AXPox6g7fdu0c_ASgsteK
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Joint+Multi-objective+Optimization+for+Radio+Access+Network+Slicing+Using+Multi-agent+Deep+Reinforcement+Learning&rft.jtitle=IEEE+transactions+on+vehicular+technology&rft.au=Zhou%2C+Guorong&rft.au=Zhao%2C+Liqiang&rft.au=Zheng%2C+Gan&rft.au=Xie%2C+Zhijie&rft.date=2023-09-01&rft.pub=IEEE&rft.issn=0018-9545&rft.spage=1&rft.epage=16&rft_id=info:doi/10.1109%2FTVT.2023.3268671&rft.externalDocID=10106015
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9545&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9545&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9545&client=summon