Joint Quantum Reinforcement Learning and Neural Myerson Auction for High-Quality Digital-Twin Services in Multitier Networks

In order to build realistic digital-twin systems, this article proposes a novel two-stage algorithm for high-quality digital-twin services in cloud-assisted multitier networks. In our proposed algorithm, the first stage is quantum multiagent reinforcement learning (QMARL)-based scheduling for differ...

Full description

Saved in:
Bibliographic Details
Published in:IEEE internet of things journal Vol. 12; no. 13; pp. 23722 - 23735
Main Authors: Park, Soohyun, Seon Kim, Gyu, Kim, Joongheon
Format: Journal Article
Language:English
Published: Piscataway IEEE 01.07.2025
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Age-of-Information (AoI)
Algorithms
Artificial intelligence
auction
Cloud computing
Computation
Deep learning
Digital twins
digital-twin
Encoding
Internet of Things
Internet of Things (IoT)
Multiagent systems
Quality control
quantum reinforcement learning (QRL)
Qubit
Reinforcement learning
Scheduling
Scheduling algorithms
Segments
Training
Virtual reality
ISSN:2327-4662, 2327-4662
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract In order to build realistic digital-twin systems, this article proposes a novel two-stage algorithm for high-quality digital-twin services in cloud-assisted multitier networks. In our proposed algorithm, the first stage is quantum multiagent reinforcement learning (QMARL)-based scheduling for differentiated quality control of individual segments of digital-twin virtual objects in our cloud. As the number of segments selected by each edge increases, the edge's action dimension expands exponentially, posing significant challenges to learning with conventional MARL. To solve this problem, the quantum-inspired MARL-based scheduler is considered in order to reduce the scheduling action dimensions into a logarithmic-scale. For the scheduling formulation, age-of-information (AoI) is also considered for low-latency high-quality digital-twin services. Additionally, the second stage is for the fast and seamless distribution of differentiated quality-controlled segments of virtual objects. For this objective, each user requests its desired segments and one of nearby edges is selected. Among various approaches, this second stage considers second price auction for truthful and distributed computation. Furthermore, low-complexity computation can be realized by avoiding integer-programming-based computation which is NP-hard. The proposed two-stage algorithm achieves performance levels that are 8.33 and 1.18 times higher in terms of reward value in high dimensions and revenue, respectively, compared to other benchmarks.
AbstractList In order to build realistic digital-twin systems, this article proposes a novel two-stage algorithm for high-quality digital-twin services in cloud-assisted multitier networks. In our proposed algorithm, the first stage is quantum multiagent reinforcement learning (QMARL)-based scheduling for differentiated quality control of individual segments of digital-twin virtual objects in our cloud. As the number of segments selected by each edge increases, the edge’s action dimension expands exponentially, posing significant challenges to learning with conventional MARL. To solve this problem, the quantum-inspired MARL-based scheduler is considered in order to reduce the scheduling action dimensions into a logarithmic-scale. For the scheduling formulation, age-of-information (AoI) is also considered for low-latency high-quality digital-twin services. Additionally, the second stage is for the fast and seamless distribution of differentiated quality-controlled segments of virtual objects. For this objective, each user requests its desired segments and one of nearby edges is selected. Among various approaches, this second stage considers second price auction for truthful and distributed computation. Furthermore, low-complexity computation can be realized by avoiding integer-programming-based computation which is NP-hard. The proposed two-stage algorithm achieves performance levels that are 8.33 and 1.18 times higher in terms of reward value in high dimensions and revenue, respectively, compared to other benchmarks.
Author Seon Kim, Gyu
Kim, Joongheon
Park, Soohyun
Author_xml – sequence: 1
  givenname: Soohyun
  orcidid: 0000-0002-6556-9746
  surname: Park
  fullname: Park, Soohyun
  email: soohyun.park@sookmyung.ac.kr
  organization: Division of Computer Science, Sookmyung Women's University, Seoul, South Korea
– sequence: 2
  givenname: Gyu
  orcidid: 0000-0002-5559-9749
  surname: Seon Kim
  fullname: Seon Kim, Gyu
  email: kingdom0545@korea.ac.kr
  organization: Department of Electrical and Computer Engineering, Korea University, Seoul, South Korea
– sequence: 3
  givenname: Joongheon
  orcidid: 0000-0003-2126-768X
  surname: Kim
  fullname: Kim, Joongheon
  email: joongheon@korea.ac.kr
  organization: Department of Electrical and Computer Engineering, Korea University, Seoul, South Korea
BookMark eNpNkEtPAjEUhRujiYj8ABMXTVwP9jEtdEnwAQQkKq4nnZk7WBw62nYkJP54S3Dh6p6cfOfc5FygU9tYQOiKkj6lRN3OpstVnxEm-lwILqg6QR3G2SBJpWSn__Q56nm_IYTEWMRkB_3MGmMDfm61De0Wv4CxVeMK2EJ056CdNXaNtS3xE7RO13ixB-cbi0dtEUy8kcYTs35PYkVtwh7fmbUJuk5WO2PxK7hvU4DHUS_aOphgwMWqsGvch79EZ5WuPfT-bhe9PdyvxpNkvnycjkfzpGDpMCQ0F4qyQuXDVAkgpSiL6OhSFjqtSpUrriQopkqqoUy1pjmRpZBEqoprqoB30c2x99M1Xy34kG2a1tn4MuOMMTlgQrJI0SNVuMZ7B1X26cxWu31GSXbYOTvsnB12zv52jpnrY8YAwD9e8QFJB_wX-iN9Ew
CODEN IITJAU
Cites_doi 10.1109/MIC.2021.3059320
10.1109/JIOT.2023.3262484
10.1109/INFOCOM.2017.8057198
10.1109/ACCESS.2023.3269577
10.1145/3470442
10.1287/moor.6.1.58
10.23919/JCIN.2022.9745481
10.1109/JIOT.2023.3300447
10.1007/s10462-020-09938-y
10.1109/TMC.2024.3407883
10.1109/JSTSP.2022.3140660
10.1109/JRFID.2022.3209715
10.1109/JIOT.2023.3282908
10.23919/JCN.2022.000033
10.1109/JIOT.2023.3287032
10.1109/TNET.2015.2452272
10.1145/3583780.3615144
10.1145/3489517.3530400
10.1109/MWC.001.1900184
10.1109/TNET.2005.852876
10.1145/3583780.3615240
10.1109/JIOT.2020.2988033
10.1109/MIC.2022.3159683
10.1109/JIOT.2022.3201082
10.1109/JSAC.2022.3221993
10.1109/JSTSP.2018.2798920
10.1109/TVT.2019.2903144
10.1016/j.icte.2022.08.004
10.1109/JIOT.2023.3234911
10.1109/TSMCB.2008.925743
10.1109/JRFID.2022.3216331
10.1561/2000000118
10.1109/TBC.2013.2273598
10.1109/6GSUMMIT49458.2020.9083808
10.1109/MCOM.020.2300199
10.1109/JSAC.2021.3088689
10.1109/JRFID.2022.3212957
10.1109/MCOM.001.2000343
ContentType Journal Article
Copyright Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025
Copyright_xml – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025
DBID 97E
RIA
RIE
AAYXX
CITATION
7SC
8FD
JQ2
L7M
L~C
L~D
DOI 10.1109/JIOT.2025.3553519
DatabaseName IEEE All-Society Periodicals Package (ASPP) 2005–Present
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE Electronic Library (IEL)
CrossRef
Computer and Information Systems Abstracts
Technology Research Database
ProQuest Computer Science Collection
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts – Academic
Computer and Information Systems Abstracts Professional
DatabaseTitle CrossRef
Computer and Information Systems Abstracts
Technology Research Database
Computer and Information Systems Abstracts – Academic
Advanced Technologies Database with Aerospace
ProQuest Computer Science Collection
Computer and Information Systems Abstracts Professional
DatabaseTitleList Computer and Information Systems Abstracts
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISSN 2327-4662
EndPage 23735
ExternalDocumentID 10_1109_JIOT_2025_3553519
10937047
Genre orig-research
GrantInformation_xml – fundername: Ministry of Science and ICT (MSIT), South Korea, through the Information Technology Research Center (ITRC) Support Program supervised by the Institute for Information and Communications Technology Planning and Evaluation (IITP)
  grantid: IITP-2024-RS-2024-00436887
– fundername: MSIT through SW Star Laboratory for Quantum AI Empowered Second-Life Platform Technology
  grantid: RS-2024-00439803
– fundername: IITP Grant
GroupedDBID 0R~
6IK
97E
AAJGR
AARMG
AASAJ
AAWTH
ABAZT
ABJNI
ABQJQ
ABVLG
AGQYO
AHBIQ
AKJIK
AKQYR
ALMA_UNASSIGNED_HOLDINGS
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
EBS
IFIPE
IPLJI
JAVBF
M43
OCL
PQQKQ
RIA
RIE
AAYXX
CITATION
7SC
8FD
JQ2
L7M
L~C
L~D
ID FETCH-LOGICAL-c248t-1b5912c9b8495e0d5dcb59ad6ca4fd9b9396e929d1aed4aa1b06d56069f3a19e3
IEDL.DBID RIE
ISICitedReferencesCount 0
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001513138500024&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 2327-4662
IngestDate Thu Nov 20 15:51:35 EST 2025
Sat Nov 29 07:49:10 EST 2025
Wed Aug 27 01:46:12 EDT 2025
IsPeerReviewed false
IsScholarly true
Issue 13
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
https://doi.org/10.15223/policy-029
https://doi.org/10.15223/policy-037
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c248t-1b5912c9b8495e0d5dcb59ad6ca4fd9b9396e929d1aed4aa1b06d56069f3a19e3
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0000-0003-2126-768X
0000-0002-5559-9749
0000-0002-6556-9746
PQID 3222672562
PQPubID 2040421
PageCount 14
ParticipantIDs ieee_primary_10937047
proquest_journals_3222672562
crossref_primary_10_1109_JIOT_2025_3553519
PublicationCentury 2000
PublicationDate 2025-07-01
PublicationDateYYYYMMDD 2025-07-01
PublicationDate_xml – month: 07
  year: 2025
  text: 2025-07-01
  day: 01
PublicationDecade 2020
PublicationPlace Piscataway
PublicationPlace_xml – name: Piscataway
PublicationTitle IEEE internet of things journal
PublicationTitleAbbrev JIoT
PublicationYear 2025
Publisher IEEE
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml – name: IEEE
– name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References ref13
ref35
ref12
ref34
ref15
ref37
ref14
ref36
ref11
You (ref30)
ref33
ref10
ref32
ref2
ref1
ref17
ref39
ref16
ref38
ref19
ref18
ref24
ref23
ref26
ref25
ref20
ref41
ref22
ref21
ref28
ref27
ref29
ref8
Sill (ref31)
Crooks (ref40) 2019
ref7
ref9
ref4
ref3
ref6
ref5
References_xml – ident: ref6
  doi: 10.1109/MIC.2021.3059320
– ident: ref21
  doi: 10.1109/JIOT.2023.3262484
– ident: ref27
  doi: 10.1109/INFOCOM.2017.8057198
– ident: ref20
  doi: 10.1109/ACCESS.2023.3269577
– ident: ref29
  doi: 10.1145/3470442
– ident: ref26
  doi: 10.1287/moor.6.1.58
– ident: ref17
  doi: 10.23919/JCIN.2022.9745481
– ident: ref4
  doi: 10.1109/JIOT.2023.3300447
– ident: ref37
  doi: 10.1007/s10462-020-09938-y
– ident: ref35
  doi: 10.1109/TMC.2024.3407883
– ident: ref7
  doi: 10.1109/JSTSP.2022.3140660
– ident: ref19
  doi: 10.1109/JRFID.2022.3209715
– ident: ref24
  doi: 10.1109/JIOT.2023.3282908
– ident: ref41
  doi: 10.23919/JCN.2022.000033
– year: 2019
  ident: ref40
  article-title: Gradients of parameterized quantum gates using the parameter-shift rule and gate decomposition
  publication-title: arXiv:1905.13311
– ident: ref3
  doi: 10.1109/JIOT.2023.3287032
– ident: ref36
  doi: 10.1109/TNET.2015.2452272
– start-page: 661
  volume-title: Proc. Conf. Neural Inf. Process. Syst. (NIPS)
  ident: ref31
  article-title: Monotonic networks
– ident: ref25
  doi: 10.1145/3583780.3615144
– ident: ref15
  doi: 10.1145/3489517.3530400
– ident: ref10
  doi: 10.1109/MWC.001.1900184
– start-page: 2981
  volume-title: Proc. Conf. Neural Inf. Process. Syst. (NIPS)
  ident: ref30
  article-title: Deep lattice networks and partial monotonic functions
– ident: ref34
  doi: 10.1109/TNET.2005.852876
– ident: ref1
  doi: 10.1145/3583780.3615240
– ident: ref9
  doi: 10.1109/JIOT.2020.2988033
– ident: ref2
  doi: 10.1109/MIC.2022.3159683
– ident: ref5
  doi: 10.1109/JIOT.2022.3201082
– ident: ref13
  doi: 10.1109/JSAC.2022.3221993
– ident: ref8
  doi: 10.1109/JSTSP.2018.2798920
– ident: ref28
  doi: 10.1109/TVT.2019.2903144
– ident: ref39
  doi: 10.1016/j.icte.2022.08.004
– ident: ref23
  doi: 10.1109/JIOT.2023.3234911
– ident: ref33
  doi: 10.1109/TSMCB.2008.925743
– ident: ref16
  doi: 10.1109/JRFID.2022.3216331
– ident: ref38
  doi: 10.1561/2000000118
– ident: ref32
  doi: 10.1109/TBC.2013.2273598
– ident: ref11
  doi: 10.1109/6GSUMMIT49458.2020.9083808
– ident: ref22
  doi: 10.1109/MCOM.020.2300199
– ident: ref14
  doi: 10.1109/JSAC.2021.3088689
– ident: ref18
  doi: 10.1109/JRFID.2022.3212957
– ident: ref12
  doi: 10.1109/MCOM.001.2000343
SSID ssj0001105196
Score 2.351277
Snippet In order to build realistic digital-twin systems, this article proposes a novel two-stage algorithm for high-quality digital-twin services in cloud-assisted...
SourceID proquest
crossref
ieee
SourceType Aggregation Database
Index Database
Publisher
StartPage 23722
SubjectTerms Age-of-Information (AoI)
Algorithms
Artificial intelligence
auction
Cloud computing
Computation
Deep learning
Digital twins
digital-twin
Encoding
Internet of Things
Internet of Things (IoT)
Multiagent systems
Quality control
quantum reinforcement learning (QRL)
Qubit
Reinforcement learning
Scheduling
Scheduling algorithms
Segments
Training
Virtual reality
Title Joint Quantum Reinforcement Learning and Neural Myerson Auction for High-Quality Digital-Twin Services in Multitier Networks
URI https://ieeexplore.ieee.org/document/10937047
https://www.proquest.com/docview/3222672562
Volume 12
WOSCitedRecordID wos001513138500024&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVIEE
  databaseName: IEEE Electronic Library (IEL)
  customDbUrl:
  eissn: 2327-4662
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0001105196
  issn: 2327-4662
  databaseCode: RIE
  dateStart: 20140101
  isFulltext: true
  titleUrlDefault: https://ieeexplore.ieee.org/
  providerName: IEEE
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NS8MwFA9uePDi_Jg4nZKDJ6Fbk7Zpcxzq0KHzgym7leajowc72Yci-Mf7knSoiAdvobyGkl-T95H3fg-hE50A0kHCPMJkBA4K0x7PghwACYXKw1jHtur96ToeDpPxmN9Vxeq2FkZrbZPPdMcM7V2-msqlCZV1DfVR7IdxDdXimLlira-ACjHWCKtuLkG0O7i6HYEHSKMOKFXTiO6H7rHNVH6dwFat9Bv__KAttFnZj7jnAN9Ga7rcQY1VbwZcbdVd9DGYFuUC3y9h5ZbP-EFbhlRpg4G4IlWd4KxU2NBzwIw3zvbGPUcni0EamyQQz7FsvOPzYmIajHijt6LEqyMGw9jV8IJ6halsTvm8iR77F6OzS6_qtOBJGiYLj4iIEyq5SMBf0r6KlIQnmWIyC3PFBQ8402BIKZJpFWYZET5TYCsxngcZ4TrYQ_VyWup9hCNwkaiMBOVEhgnlgtBc0EAGJNTcF1ELna4wSF8coUZqHRGfpwaw1ACWVoC1UNMs-jdBt94t1F7BllZ7bp6aOyMWgwlHD_547RBtmNldtm0b1RezpT5C6_J1Ucxnx_Z3-gTjf8sD
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NT9swFH8aHRK70PGlFdjwgRNSSuw4TnxEG1XLSvlQQdyi-COoh6UTbUGT9sfzbKcChHbYzYpsJ_Iv9vvwe78HcGhzRDrJRUSFTtFAETaSZVIhIFyZimc281nvt8NsNMrv7uRlk6zuc2GstT74zHZd09_lm6leOFfZsaM-ymKercDHlHMWh3StF5cKdfqIaO4usfPx2eBijDYgS7soVl0pujfSx5dTeXcGe8HSa__nJ32G9UaDJCcB8g34YOtNaC-rM5Bms27B37PppJ6TqwWu3eIXubaeI1V7dyBpaFXvSVkb4gg6cMbzoH2Tk0AoS7A3cWEgUeDZ-EN-TO5diZFo_DSpyfKQIdgOWbwoYHEqH1U-24ab3un4ez9qai1EmvF8HlGVSsq0VDlaTDY2qdH4pDRCl7wyUslECouqlKGlNbwsqYqFQW1JyCopqbTJDrTqaW2_AEnRSGI6VUxSzXMmFWWVYolOKLcyVmkHjpYYFL8DpUbhTZFYFg6wwgFWNIB1YNst-quOYb07sL-ErWh23axwt0YiQyWO7f5j2AGs9cfnw2I4GP3cg0_uTSH2dh9a84eF_Qqr-nE-mT1887_WM5hdzko
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Joint+Quantum+Reinforcement+Learning+and+Neural+Myerson+Auction+for+High-Quality+Digital-Twin+Services+in+Multitier+Networks&rft.jtitle=IEEE+internet+of+things+journal&rft.au=Park%2C+Soohyun&rft.au=Seon+Kim%2C+Gyu&rft.au=Kim%2C+Joongheon&rft.date=2025-07-01&rft.pub=IEEE&rft.eissn=2327-4662&rft.volume=12&rft.issue=13&rft.spage=23722&rft.epage=23735&rft_id=info:doi/10.1109%2FJIOT.2025.3553519&rft.externalDocID=10937047
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2327-4662&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2327-4662&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2327-4662&client=summon