Joint Multi-objective Optimization for Radio Access Network Slicing Using Multi-agent Deep Reinforcement Learning

Radio access network (RAN) slices can provide various customized services for next-generation wireless networks. Thus, multiple performance metrics of different types of RAN slices need to be jointly optimized. However, existing efforts in multi-objective optimization problem (MOOP) for RAN slicing...

Full description

Saved in:

Bibliographic Details
Published in:	IEEE transactions on vehicular technology Vol. 72; no. 9; pp. 1 - 16
Main Authors:	Zhou, Guorong, Zhao, Liqiang, Zheng, Gan, Xie, Zhijie, Song, Shenghui, Chen, Kwang-Cheng
Format:	Journal Article
Language:	English
Published:	New York IEEE 01.09.2023 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:	Algorithms Delays Heuristic algorithms Hierarchies multi-agent deep reinforcement learning multi-objective optimization Multiagent systems Multiple objective analysis Network slicing non-scalarization Optimization Pareto optimization Pareto optimum Performance measurement Polynomials Radio access network slicing rank voting method Resource management Throughput Wireless networks
ISSN:	0018-9545, 1939-9359
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Abstract	Radio access network (RAN) slices can provide various customized services for next-generation wireless networks. Thus, multiple performance metrics of different types of RAN slices need to be jointly optimized. However, existing efforts in multi-objective optimization problem (MOOP) for RAN slicing are only in the scalar form, which is difficult to achieve simultaneous optimization. In this paper, we consider a non-scalar MOOP for RAN slicing with three types of slices, i.e. , the high-bandwidth slice, the low-delay slice, and the wide-coverage slice over the same underlying physical network. We jointly optimize the throughput, the transmission delay, and the coverage area by user-oriented dynamic virtual base stations (vBSs)' deployment, and sub-channel and power allocation. An improved multi-agent deep deterministic policy gradient (IMADDPG) algorithm, having the characteristics of centralized training and distributed execution, is proposed to solve the above non-deterministic polynomial-time hard (NP-hard) problem. The rank voting method is introduced in the inference process to obtain near-Pareto optimal solutions. Simulation results verify that the proposed scheme can ensure better performance than the traditional scalar utility method and other benchmark algorithms. The proposed scheme has the advantage of flexibly approaching any point of the Pareto boundary, while the traditional scalar method only subjectively approaches one of the Pareto optimal solutions. Furthermore, our proposal strikes a compelling tradeoff among three types of RAN slices due to the non-dominance between Pareto optimal solutions.
AbstractList	Radio access network (RAN) slices can provide various customized services for next-generation wireless networks. Thus, multiple performance metrics of different types of RAN slices need to be jointly optimized. However, existing efforts in multi-objective optimization problem (MOOP) for RAN slicing are only in the scalar form, which is difficult to achieve simultaneous optimization. In this paper, we consider a non-scalar MOOP for RAN slicing with three types of slices, i.e., the high-bandwidth slice, the low-delay slice, and the wide-coverage slice over the same underlying physical network. We jointly optimize the throughput, the transmission delay, and the coverage area by user-oriented dynamic virtual base stations (vBSs)’ deployment, and sub-channel and power allocation. An improved multi-agent deep deterministic policy gradient (IMADDPG) algorithm, having the characteristics of centralized training and distributed execution, is proposed to solve the above non-deterministic polynomial-time hard (NP-hard) problem. The rank voting method is introduced in the inference process to obtain near-Pareto optimal solutions. Simulation results verify that the proposed scheme can ensure better performance than the traditional scalar utility method and other benchmark algorithms. The proposed scheme has the advantage of flexibly approaching any point of the Pareto boundary, while the traditional scalar method only subjectively approaches one of the Pareto optimal solutions. Furthermore, our proposal strikes a compelling tradeoff among three types of RAN slices due to the non-dominance between Pareto optimal solutions.
Author	Zheng, Gan Song, Shenghui Xie, Zhijie Zhao, Liqiang Chen, Kwang-Cheng Zhou, Guorong
Author_xml	– sequence: 1 givenname: Guorong orcidid: 0000-0001-9430-1397 surname: Zhou fullname: Zhou, Guorong organization: State Key Laboratory of Integrated Service Networks, Xidian University, Xi'an, China – sequence: 2 givenname: Liqiang orcidid: 0000-0002-3374-6066 surname: Zhao fullname: Zhao, Liqiang organization: State Key Laboratory of Integrated Service Networks, Xidian University, Xi'an, China – sequence: 3 givenname: Gan orcidid: 0000-0001-8457-6477 surname: Zheng fullname: Zheng, Gan organization: School of Engineering, University of Warwick, Coventry, U.K – sequence: 4 givenname: Zhijie surname: Xie fullname: Xie, Zhijie organization: Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong – sequence: 5 givenname: Shenghui orcidid: 0000-0001-6316-8415 surname: Song fullname: Song, Shenghui organization: Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong – sequence: 6 givenname: Kwang-Cheng orcidid: 0000-0002-1024-6106 surname: Chen fullname: Chen, Kwang-Cheng organization: Department of Electrical Engineering, University of South Florida, Tampa, FL, USA
BookMark	eNp9kE1PAjEURRujiYjuXbho4nqw7bSd6ZL4bVASBLeTtrwhRZhiWzT66x2EhXHh5jWvuee-5Byh_cY3gNApJT1KiboYv4x7jLC8lzNZyoLuoQ5VucpULtQ-6hBCy0wJLg7RUYzzduVc0Q56e_CuSfhxvUgu82YONrl3wMNVckv3pZPzDa59wCM9dR73rYUY8ROkDx9e8fPCWdfM8CRu5rZDz6DtuwJY4RG4pmUtLDdfA9ChaXPH6KDWiwgnu7eLJjfX48u7bDC8vb_sDzLLFEuZNsQUhheSUas4sGkpSWFyQZQhYIUoeF6XjJqpoLVQNTeac2ZkqaSYAqlZ3kXn295V8G9riKma-3Vo2pMVK6XkihBRtimyTdngYwxQV6vgljp8VpRUG7FVK7baiK12YltE_kGsSz-mUtBu8R94tgUdAPy6Q4kkVOTfOLyIaQ
CODEN	ITVTAB
CitedBy_id	crossref_primary_10_1109_COMST_2023_3338153 crossref_primary_10_1109_LWC_2024_3365161 crossref_primary_10_1109_TPDS_2023_3310013 crossref_primary_10_1109_ACCESS_2025_3538546 crossref_primary_10_1109_TNSM_2024_3476480 crossref_primary_10_1109_TSC_2025_3586152 crossref_primary_10_1109_TVT_2024_3431878 crossref_primary_10_1007_s11082_023_05796_4 crossref_primary_10_1109_ACCESS_2023_3296851 crossref_primary_10_1109_TGCN_2024_3397459 crossref_primary_10_54392_irjmt24324 crossref_primary_10_1109_JIOT_2023_3319130 crossref_primary_10_1109_JIOT_2024_3416157 crossref_primary_10_1109_TCOMM_2023_3335414
Cites_doi	10.1109/TWC.2017.2789294 10.1109/TVT.2022.3202689 10.1109/TWC.2021.3123500 10.1109/TNSE.2018.2876918 10.1109/SPAWC.2017.8227791 10.1109/JIOT.2018.2888543 10.1609/aaai.v32i1.11694 10.1109/MSP.2014.2330661 10.1109/JIOT.2021.3068518 10.1109/COMST.2016.2610578 10.1109/TWC.2017.2696000 10.1109/ACCESS.2019.2902432 10.1109/TCOMM.2022.3211083 10.1109/TVT.2019.2952216 10.1109/JSAC.2019.2959185 10.1109/MVT.2020.3015184 10.1109/TVT.2019.2896586 10.1109/TWC.2017.2725836 10.1109/TMC.2019.2930059 10.1162/106365600568158 10.1109/TWC.2020.2965927 10.1109/TVT.2021.3095901 10.1109/JSAC.2019.2933893 10.1109/JIOT.2021.3111644 10.1109/TNSM.2019.2899609 10.1109/JSAC.2020.2986869 10.1109/TVT.2007.912960 10.1109/COMST.2018.2815638 10.1109/ACCESS.2018.2822398 10.1109/ACCESS.2020.3036416 10.1109/TVT.2017.2738024 10.1109/JIOT.2021.3089823 10.1109/TKDE.2012.73 10.1109/MCOM.2014.6766097 10.1109/JIOT.2020.2997342 10.1109/MCOM.2017.1600951 10.1109/COMST.2020.2965856 10.1109/ACCESS.2018.2881964 10.1109/TVT.2020.2972999 10.1038/nature24270 10.1109/MVT.2021.3085511 10.1016/j.neucom.2017.02.096 10.1109/OJVT.2021.3095467 10.1109/TCOMM.2021.3090423 10.1109/PESGM.2017.8274155 10.1109/TNSM.2019.2945254 10.1109/ACCESS.2019.2909670
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023
DBID	97E RIA RIE AAYXX CITATION 7SP 8FD FR3 KR7 L7M
DOI	10.1109/TVT.2023.3268671
DatabaseName	IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Electronics & Communications Abstracts Technology Research Database Engineering Research Database Civil Engineering Abstracts Advanced Technologies Database with Aerospace
DatabaseTitle	CrossRef Civil Engineering Abstracts Engineering Research Database Technology Research Database Advanced Technologies Database with Aerospace Electronics & Communications Abstracts
DatabaseTitleList	Civil Engineering Abstracts
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	1939-9359
EndPage	16
ExternalDocumentID	10_1109_TVT_2023_3268671 10106015
Genre	orig-research
GrantInformation_xml	– fundername: Key-Area Research and Development Program of Guangdong Province grantid: 2020B0101120003 – fundername: National Natural Science Foundation of China; NSFC grantid: 62071352 funderid: 10.13039/501100001809 – fundername: National Key R&D Program of China grantid: 2020YFB1807700; 2020YFB1806404 – fundername: Key Research and Development Projects of Shaanxi Province; Key Research and Development Program of Shaanxi grantid: 2022KWZ- funderid: 10.13039/501100015401 – fundername: The Hong Kong University of Science and Technology (HKUST) Startup Fund grantid: R9249 – fundername: Higher Education Discipline Innovation Project; 111 Project grantid: B08038 funderid: 10.13039/501100013314
GroupedDBID	-~X .DC 0R~ 29I 4.4 5GY 6IK 97E AAIKC AAJGR AAMNW AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK ACNCT AENEX AGQYO AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS HZ~ IFIPE IPLJI JAVBF LAI M43 MS~ O9- OCL P2P RIA RIE RNS RXW TAE TN5 3EH 5VS AAYXX AETIX AGSQL AI. AIBXA ALLEH CITATION EJD H~9 IAAWW IBMZZ ICLAB IFJZH VH1 7SP 8FD FR3 KR7 L7M
ID	FETCH-LOGICAL-c292t-ab0b7b47621c94e2d8607b3509b0ec55743f821bd51f59f4ba442b68965de0f23
IEDL.DBID	RIE
ISICitedReferencesCount	15
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001103676800058&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0018-9545
IngestDate	Mon Jun 30 10:15:01 EDT 2025 Tue Nov 18 22:20:45 EST 2025 Sat Nov 29 02:59:08 EST 2025 Wed Aug 27 02:14:16 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	9
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c292t-ab0b7b47621c94e2d8607b3509b0ec55743f821bd51f59f4ba442b68965de0f23
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0001-9430-1397 0000-0002-1024-6106 0000-0001-8457-6477 0000-0001-6316-8415 0000-0002-3374-6066 0000-0002-2003-0472
PQID	2866490058
PQPubID	85454
PageCount	16
ParticipantIDs	crossref_primary_10_1109_TVT_2023_3268671 crossref_citationtrail_10_1109_TVT_2023_3268671 proquest_journals_2866490058 ieee_primary_10106015
PublicationCentury	2000
PublicationDate	2023-09-01
PublicationDateYYYYMMDD	2023-09-01
PublicationDate_xml	– month: 09 year: 2023 text: 2023-09-01 day: 01
PublicationDecade	2020
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	IEEE transactions on vehicular technology
PublicationTitleAbbrev	TVT
PublicationYear	2023
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 ref12 ref15 ref14 ref11 ref10 ref17 ref16 ref19 ref18 ref46 ref45 ref48 ref47 ref42 ref41 ref44 ref43 ref49 ref8 ref7 ref9 ref4 ref3 ref6 ref5 ref35 ref34 ref37 ref36 ref31 ref30 ref33 ref2 ref1 ref39 ref38 ref24 ref23 ref26 ref25 ref20 ref22 ref21 ref28 ref27 ref29 bertsekas (ref40) 1992 lowe (ref32) 0
References_xml	– ident: ref5 doi: 10.1109/TWC.2017.2789294 – ident: ref49 doi: 10.1109/TVT.2022.3202689 – ident: ref42 doi: 10.1109/TWC.2021.3123500 – ident: ref23 doi: 10.1109/TNSE.2018.2876918 – start-page: 6379 year: 0 ident: ref32 article-title: Multi-agent actor-critic for mixed cooperative-competitive environments publication-title: Proc Adv Neural Inf Process Syst – ident: ref7 doi: 10.1109/SPAWC.2017.8227791 – year: 1992 ident: ref40 publication-title: Data Networks – ident: ref19 doi: 10.1109/JIOT.2018.2888543 – ident: ref48 doi: 10.1609/aaai.v32i1.11694 – ident: ref21 doi: 10.1109/MSP.2014.2330661 – ident: ref20 doi: 10.1109/JIOT.2021.3068518 – ident: ref15 doi: 10.1109/COMST.2016.2610578 – ident: ref41 doi: 10.1109/TWC.2017.2696000 – ident: ref30 doi: 10.1109/ACCESS.2019.2902432 – ident: ref34 doi: 10.1109/TCOMM.2022.3211083 – ident: ref38 doi: 10.1109/TVT.2019.2952216 – ident: ref9 doi: 10.1109/JSAC.2019.2959185 – ident: ref13 doi: 10.1109/MVT.2020.3015184 – ident: ref27 doi: 10.1109/TVT.2019.2896586 – ident: ref25 doi: 10.1109/TWC.2017.2725836 – ident: ref22 doi: 10.1109/TMC.2019.2930059 – ident: ref31 doi: 10.1162/106365600568158 – ident: ref4 doi: 10.1109/TWC.2020.2965927 – ident: ref3 doi: 10.1109/TVT.2021.3095901 – ident: ref39 doi: 10.1109/JSAC.2019.2933893 – ident: ref2 doi: 10.1109/JIOT.2021.3111644 – ident: ref24 doi: 10.1109/TNSM.2019.2899609 – ident: ref33 doi: 10.1109/JSAC.2020.2986869 – ident: ref29 doi: 10.1109/TVT.2007.912960 – ident: ref1 doi: 10.1109/COMST.2018.2815638 – ident: ref17 doi: 10.1109/ACCESS.2018.2822398 – ident: ref35 doi: 10.1109/ACCESS.2020.3036416 – ident: ref8 doi: 10.1109/TVT.2017.2738024 – ident: ref36 doi: 10.1109/JIOT.2021.3089823 – ident: ref28 doi: 10.1109/TKDE.2012.73 – ident: ref26 doi: 10.1109/MCOM.2014.6766097 – ident: ref18 doi: 10.1109/JIOT.2020.2997342 – ident: ref37 doi: 10.1109/MCOM.2017.1600951 – ident: ref14 doi: 10.1109/COMST.2020.2965856 – ident: ref43 doi: 10.1109/ACCESS.2018.2881964 – ident: ref6 doi: 10.1109/TVT.2020.2972999 – ident: ref44 doi: 10.1038/nature24270 – ident: ref10 doi: 10.1109/MVT.2021.3085511 – ident: ref46 doi: 10.1016/j.neucom.2017.02.096 – ident: ref11 doi: 10.1109/OJVT.2021.3095467 – ident: ref47 doi: 10.1109/TCOMM.2021.3090423 – ident: ref12 doi: 10.1109/PESGM.2017.8274155 – ident: ref16 doi: 10.1109/TNSM.2019.2945254 – ident: ref45 doi: 10.1109/ACCESS.2019.2909670
SSID	ssj0014491
Score	2.487317
Snippet	Radio access network (RAN) slices can provide various customized services for next-generation wireless networks. Thus, multiple performance metrics of...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	1
SubjectTerms	Algorithms Delays Heuristic algorithms Hierarchies multi-agent deep reinforcement learning multi-objective optimization Multiagent systems Multiple objective analysis Network slicing non-scalarization Optimization Pareto optimization Pareto optimum Performance measurement Polynomials Radio access network slicing rank voting method Resource management Throughput Wireless networks
Title	Joint Multi-objective Optimization for Radio Access Network Slicing Using Multi-agent Deep Reinforcement Learning
URI	https://ieeexplore.ieee.org/document/10106015 https://www.proquest.com/docview/2866490058
Volume	72
WOSCitedRecordID	wos001103676800058&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1939-9359 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014491 issn: 0018-9545 databaseCode: RIE dateStart: 19670101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PT8IwFG6UeNCDPzGiaHrw4qGwdt2PHolKjDFoEA23pe06gsENYfj323aFYIwm3nZom2Xf3uvXvve-B8AlZpEKJKYoEKlENJMR4pxnKCZGnSvVDFxYdf2HqNeLh0P25IrVbS2MUsomn6mWebSx_LSQC3NVpi0cG_mQYBNsRlFYFWutQgaUuvZ4WFuw5gXLmKTH2oPXQcu0CW9prmL03L7tQbapyg9PbLeX7t4_X2wf7DoeCTsV8AdgQ-WHYGdNXfAIfNwX47yEtsQWFeKtcm3wUTuJd1d9CTVlhX2ejgvYsZ0TYa9KC4fPExNyH0GbUuDW4KYMC94oNYV9ZRVXpb1chE6kdVQHL93bwfUdch0WkCSMlIgLT0SCaoeIJaOKpHHoRcLXJEJ4SgaBphdZTLBIA5wFLKOCU0pEGDMNo_Iy4h-DWl7k6gRALDS3YJJy35caX6OMJ3nMzZGMZCz1G6C9_OaJdPLjpgvGJLHHEI8lGqXEoJQ4lBrgajVjWklv_DG2blBZG1cB0gDNJa6JM855QuIwpMw0VDz9ZdoZ2DarV7lkTVArZwt1DrbkZzmezy7sf_cFVfHVXA
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3JTsMwEB2xScCBtYhCAR-4cEhJHKeJj4hFLCWgEhC3yHacqqgkpRS-H9txqyIEErcc7CTys8fPnpk3AIceDWUgPOIEPBMOyUXoMMZyJ8JanStTDJwbdf12GMfR8zO9t8nqJhdGSmmCz2RTPxpfflaKD31Vpla4p-VDglmYDwjBbpWuNXEaEGIL5HlqDStmMPZKuvQ4eUqaulB4U7EVrej2bRcyZVV-2GKzwVys_vPX1mDFMkl0UkG_DjOy2IDlKX3BTXi7LnvFCJkkW6fkL5VxQ3fKTLza_EukSCvqsKxXohNTOxHFVWA4euhrp3sXmaAC-w6mE7HQmZQD1JFGc1WY60VkZVq7NXi8OE9OLx1bY8ERmOKRw7jLQ06USfQEJRJnUcsNua9oBHelCAJFMPIIezwLvDygOeFMDTxvRVQBKd0c-1swV5SF3AbkccUuqCDM94VCWGvjCRYxfSjDOc38OhyPxzwVVoBc18Hop-Yg4tJUoZRqlFKLUh2OJj0GlfjGH21rGpWpdhUgdWiMcU3t8nxPcdRqEapLKu780u0AFi-T23bavopvdmFJf6mKLGvA3Gj4IfdgQXyOeu_DfTMHvwCXKtij
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Joint+Multi-Objective+Optimization+for+Radio+Access+Network+Slicing+Using+Multi-Agent+Deep+Reinforcement+Learning&rft.jtitle=IEEE+transactions+on+vehicular+technology&rft.au=Zhou%2C+Guorong&rft.au=Zhao%2C+Liqiang&rft.au=Zheng%2C+Gan&rft.au=Xie%2C+Zhijie&rft.date=2023-09-01&rft.issn=0018-9545&rft.eissn=1939-9359&rft.volume=72&rft.issue=9&rft.spage=11828&rft.epage=11843&rft_id=info:doi/10.1109%2FTVT.2023.3268671&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TVT_2023_3268671
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9545&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9545&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9545&client=summon