Delay-Tolerant OCO With Long-Term Constraints: Algorithm and Its Application to Network Resource Allocation

We consider online convex optimization (OCO) with multi-slot feedback delay. An agent selects a sequence of online decisions to minimize the accumulation of time-varying convex loss functions, subject to short-term and long-term constraints that may be time-varying. Both the convex loss function and...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE/ACM transactions on networking Jg. 31; H. 1; S. 147 - 163
Hauptverfasser: Wang, Juncheng, Dong, Min, Liang, Ben, Boudreau, Gary, Abou-Zeid, Hatem
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York IEEE 01.02.2023
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:
Algorithms
Benchmark testing
Benchmarks
Computational geometry
constraint violation
Constraints
Convexity
Decision theory
Delay
Delays
dynamic regret
Feedback
Heuristic algorithms
long-term constraint
Mathematical analysis
multi-slot delay
Online convex optimization
online network resource allocation
Optimization
Regularization
Resource allocation
Resource management
Upper bounds
Wireless communication
ISSN:1063-6692, 1558-2566
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Abstract We consider online convex optimization (OCO) with multi-slot feedback delay. An agent selects a sequence of online decisions to minimize the accumulation of time-varying convex loss functions, subject to short-term and long-term constraints that may be time-varying. Both the convex loss function and the long-term constraint function may experience multiple time slots of feedback delay to be received by the agent. Existing works on OCO under this general setting has focused on the static regret, which measures the gap of losses between an online decision sequence and a time-invariant static offline benchmark. In this work, besides the static regret, we also consider a more practically meaningful metric, the dynamic regret, where the benchmark is a time-varying online optimal decision sequence. We propose an efficient algorithm, termed Delay-Tolerant Constrained-OCO (DTC-OCO), which uses a novel double regularization together with a new penalty mechanism on the long-term constraint violation, to tackle the asynchrony between information feedback and decision updates. We obtain upper bounds for its static regret, dynamic regret, and constraint violation, proving that they are sublinear under mild conditions. Furthermore, we consider a variation of DTC-OCO with multi-step gradient descent, and show it provides improved dynamic regret and constraint violation bounds for strongly convex loss functions. For numerical demonstration, we apply DTC-OCO to a general network resource allocation problem. Our simulation results suggest substantial performance gain by DTC-OCO over the current best alternative.
AbstractList We consider online convex optimization (OCO) with multi-slot feedback delay. An agent selects a sequence of online decisions to minimize the accumulation of time-varying convex loss functions, subject to short-term and long-term constraints that may be time-varying. Both the convex loss function and the long-term constraint function may experience multiple time slots of feedback delay to be received by the agent. Existing works on OCO under this general setting has focused on the static regret, which measures the gap of losses between an online decision sequence and a time-invariant static offline benchmark. In this work, besides the static regret, we also consider a more practically meaningful metric, the dynamic regret, where the benchmark is a time-varying online optimal decision sequence. We propose an efficient algorithm, termed Delay-Tolerant Constrained-OCO (DTC-OCO), which uses a novel double regularization together with a new penalty mechanism on the long-term constraint violation, to tackle the asynchrony between information feedback and decision updates. We obtain upper bounds for its static regret, dynamic regret, and constraint violation, proving that they are sublinear under mild conditions. Furthermore, we consider a variation of DTC-OCO with multi-step gradient descent, and show it provides improved dynamic regret and constraint violation bounds for strongly convex loss functions. For numerical demonstration, we apply DTC-OCO to a general network resource allocation problem. Our simulation results suggest substantial performance gain by DTC-OCO over the current best alternative.
Author Wang, Juncheng
Abou-Zeid, Hatem
Boudreau, Gary
Dong, Min
Liang, Ben
Author_xml – sequence: 1
  givenname: Juncheng
  orcidid: 0000-0003-1375-8382
  surname: Wang
  fullname: Wang, Juncheng
  email: jcwang@ece.utoronto.ca
  organization: Department of Electrical and Computer Engineering, University of Toronto, Toronto, Canada
– sequence: 2
  givenname: Min
  orcidid: 0000-0002-7223-8865
  surname: Dong
  fullname: Dong, Min
  email: min.dong@ontariotechu.ca
  organization: Department of Electrical, Computer, and Software Engineering, Ontario Tech University, Oshawa, Canada
– sequence: 3
  givenname: Ben
  orcidid: 0000-0002-1800-1322
  surname: Liang
  fullname: Liang, Ben
  email: liang@ece.utoronto.ca
  organization: Department of Electrical and Computer Engineering, University of Toronto, Toronto, Canada
– sequence: 4
  givenname: Gary
  orcidid: 0000-0003-3539-9624
  surname: Boudreau
  fullname: Boudreau, Gary
  email: gary.boudreau@ericsson.com
  organization: Ericsson Canada, Ottawa, Canada
– sequence: 5
  givenname: Hatem
  surname: Abou-Zeid
  fullname: Abou-Zeid, Hatem
  email: hatem.abouzeid@ucalgary.ca
  organization: Ericsson Canada, Ottawa, Canada
BookMark eNp9kE1LAzEQhoMo-PkDxEvA89Z8uNnEW1nrB5QWZMXjko0T3XabrEmK-O_d2uLBg6cZmPeZeec9RvvOO0DonJIRpURdVbNJNWKEsRGnUjKZ76EjmucyY7kQ-0NPBM-EUOwQHce4IIRywsQRWt5Cp7-yyncQtEt4Xs7xS5ve8dS7t6yCsMKldzEF3boUb_C4e_NhmK-wdq_4MUU87vuuNTq13uHk8QzSpw9L_ATRr4OBgej8dnyKDqzuIpzt6gl6vptU5UM2nd8_luNpZpjiKcuZVU2huaXCUHttXq0VQkJjdCOpErTh0jZAORiuDCfWALO20FIaLgulLT9Bl9u9ffAfa4ipXgxW3HCyZkUhCnUtORlUxVZlgo8xgK1Nm358bp7takrqTbL1Jtl6k2y9S3Yg6R-yD-1Kh69_mYst0wLAr15JTmku-TfnbIff
CODEN IEANEP
CitedBy_id crossref_primary_10_26599_TST_2024_9010140
crossref_primary_10_1109_TWC_2023_3293502
crossref_primary_10_1109_TNET_2024_3495522
crossref_primary_10_1109_TON_2025_3530460
crossref_primary_10_1016_j_eswa_2025_127871
Cites_doi 10.1109/JSTSP.2018.2827302
10.1145/3393691.3394209
10.1109/NOMS.2010.5488484
10.1109/TAC.2017.2743462
10.1002/9780470512531
10.1561/2400000013
10.1109/TSP.2018.2890368
10.1109/TSP.2017.2679690
10.1287/opre.2015.1408
10.1109/JSTSP.2015.2404790
10.1109/TSP.2020.2964200
10.1109/TAC.2020.3030743
10.1109/INFOCOM42981.2021.9488698
10.1007/s10994-007-5016-8
10.1109/CDC.2016.7799379
10.1109/TSP.2017.2750109
10.1109/INFCOM.2010.5462170
10.1109/JSTSP.2018.2797423
10.1109/INFOCOM41043.2020.9155347
10.1109/TCNS.2015.2505149
10.1561/1300000001
10.1109/INFOCOM.2018.8486305
10.1007/978-3-031-79995-2
10.1561/2200000018
10.1109/TWC.2017.2690442
10.1109/TAC.2011.2161027
10.1109/TWC.2013.072513.121842
10.1109/71.993206
ContentType Journal Article
Copyright Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023
Copyright_xml – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023
DBID 97E
RIA
RIE
AAYXX
CITATION
7SC
7SP
8FD
JQ2
L7M
L~C
L~D
DOI 10.1109/TNET.2022.3188285
DatabaseName IEEE All-Society Periodicals Package (ASPP) 2005–Present
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE Electronic Library (IEL)
CrossRef
Computer and Information Systems Abstracts
Electronics & Communications Abstracts
Technology Research Database
ProQuest Computer Science Collection
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts – Academic
Computer and Information Systems Abstracts Professional
DatabaseTitle CrossRef
Technology Research Database
Computer and Information Systems Abstracts – Academic
Electronics & Communications Abstracts
ProQuest Computer Science Collection
Computer and Information Systems Abstracts
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts Professional
DatabaseTitleList
Technology Research Database
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
EISSN 1558-2566
EndPage 163
ExternalDocumentID 10_1109_TNET_2022_3188285
9831158
Genre orig-research
GrantInformation_xml – fundername: Ontario Centre of Innovation
– fundername: Ericsson Canada
– fundername: Natural Sciences and Engineering Research Council of Canada
  funderid: 10.13039/501100000038
GroupedDBID -DZ
-~X
.DC
0R~
29I
4.4
5GY
5VS
6IK
85S
8US
97E
9M8
AAJGR
AAKMM
AALFJ
AARMG
AASAJ
AAWTH
AAWTV
ABAZT
ABPPZ
ABQJQ
ABVLG
ACGFS
ACGOD
ACIWK
ACM
ADBCU
ADL
AEBYY
AEFXT
AEJOY
AENSD
AETEA
AETIX
AFWIH
AFWXC
AGQYO
AGSQL
AHBIQ
AI.
AIBXA
AIKLT
AKJIK
AKQYR
AKRVB
ALLEH
ALMA_UNASSIGNED_HOLDINGS
ATWAV
BDXCO
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CCLIF
CS3
D0L
EBS
EJD
FEDTE
GUFHI
HF~
HGAVV
HZ~
H~9
I07
ICLAB
IEDLZ
IES
IFIPE
IFJZH
IPLJI
JAVBF
LAI
LHSKQ
M43
MVM
O9-
OCL
P1C
P2P
PQQKQ
RIA
RIE
RNS
ROL
TN5
UPT
UQL
VH1
XOL
YR2
ZCA
AAYXX
CITATION
7SC
7SP
8FD
JQ2
L7M
L~C
L~D
ID FETCH-LOGICAL-c293t-52f9b7a3f16c1f4cdff668ebcab81961b38fbe13ec39c30fce2ff7a88c3879af3
IEDL.DBID RIE
ISICitedReferencesCount 10
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000829067600001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 1063-6692
IngestDate Sun Nov 09 07:38:05 EST 2025
Tue Nov 18 22:11:38 EST 2025
Sat Nov 29 03:05:26 EST 2025
Wed Aug 27 02:14:22 EDT 2025
IsPeerReviewed true
IsScholarly true
Issue 1
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
https://doi.org/10.15223/policy-029
https://doi.org/10.15223/policy-037
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c293t-52f9b7a3f16c1f4cdff668ebcab81961b38fbe13ec39c30fce2ff7a88c3879af3
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0000-0002-7223-8865
0000-0002-1800-1322
0000-0003-1375-8382
0000-0003-3539-9624
PQID 2776794830
PQPubID 32020
PageCount 17
ParticipantIDs ieee_primary_9831158
proquest_journals_2776794830
crossref_citationtrail_10_1109_TNET_2022_3188285
crossref_primary_10_1109_TNET_2022_3188285
PublicationCentury 2000
PublicationDate 2023-Feb.
2023-2-00
20230201
PublicationDateYYYYMMDD 2023-02-01
PublicationDate_xml – month: 02
  year: 2023
  text: 2023-Feb.
PublicationDecade 2020
PublicationPlace New York
PublicationPlace_xml – name: New York
PublicationTitle IEEE/ACM transactions on networking
PublicationTitleAbbrev TNET
PublicationYear 2023
Publisher IEEE
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml – name: IEEE
– name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References ref35
ref37
ref36
ref31
ref30
ref33
ref10
Zinkevich (ref4)
ref2
Zhang (ref11)
ref1
Eshraghi (ref32)
ref17
McMahan (ref21)
ref39
ref16
ref38
ref18
Yu (ref14) 2020; 21
ref24
ref23
Yu (ref15)
ref26
Langford (ref19)
ref25
Mannor (ref28) 2009; 10
ref41
ref22
Zhang (ref34)
ref27
Jadbabaie (ref7); 38
ref29
ref8
Mahdavi (ref12) 2012; 13
ref9
ref3
ref6
ref5
ref40
Jenatton (ref13)
Quanrud (ref20)
References_xml – ident: ref18
  doi: 10.1109/JSTSP.2018.2827302
– start-page: 2331
  volume-title: Proc. Adv. Neural Inf. Process. Syst. (NIPS)
  ident: ref19
  article-title: Slow learners are fast
– ident: ref16
  doi: 10.1145/3393691.3394209
– start-page: 1323
  volume-title: Proc. Adv. Neural Inf. Process. Syst. (NIPS)
  ident: ref34
  article-title: Adaptive online learning in dynamic environments
– ident: ref38
  doi: 10.1109/NOMS.2010.5488484
– ident: ref31
  doi: 10.1109/TAC.2017.2743462
– start-page: 1270
  volume-title: Proc. Adv. Neural Inf. Process. Syst. (NIPS)
  ident: ref20
  article-title: Online learning with adversarial delays
– start-page: 402
  volume-title: Proc. Int. Conf. Mach. Learn. (ICML)
  ident: ref13
  article-title: Adaptive algorithms for online convex optimization with long-term constraints
– ident: ref37
  doi: 10.1002/9780470512531
– ident: ref3
  doi: 10.1561/2400000013
– ident: ref10
  doi: 10.1109/TSP.2018.2890368
– start-page: 928
  volume-title: Proc. Intel. Conf. Mach. Learn. (ICML)
  ident: ref4
  article-title: Online convex programming and generalized infinitesimal gradient ascent
– ident: ref36
  doi: 10.1109/TSP.2017.2679690
– ident: ref23
  doi: 10.1287/opre.2015.1408
– ident: ref6
  doi: 10.1109/JSTSP.2015.2404790
– volume: 38
  start-page: 398
  volume-title: Proc. Int. Conf. Artif. Intell. Statist. (AISTATS)
  ident: ref7
  article-title: Online optimization: Competing with dynamic comparators
– ident: ref33
  doi: 10.1109/TSP.2020.2964200
– start-page: 1427
  volume-title: Proc. Adv. Neural Inf. Process. Syst. (NIPS)
  ident: ref15
  article-title: Online convex optimization with stochastic constraints
– start-page: 693
  volume-title: Proc. Adv. Neural Inf. Process. Syst. (NIPS)
  ident: ref21
  article-title: Delay-tolerant algorithms for asynchronous distributed online learning
– ident: ref22
  doi: 10.1109/TAC.2020.3030743
– ident: ref1
  doi: 10.1109/INFOCOM42981.2021.9488698
– ident: ref5
  doi: 10.1007/s10994-007-5016-8
– ident: ref8
  doi: 10.1109/CDC.2016.7799379
– ident: ref17
  doi: 10.1109/TSP.2017.2750109
– ident: ref25
  doi: 10.1109/INFCOM.2010.5462170
– ident: ref9
  doi: 10.1109/JSTSP.2018.2797423
– start-page: 1
  volume-title: Proc. Adv. Neural Inf. Process. Syst. (NIPS)
  ident: ref11
  article-title: Improved dynamic regret for non-degenerate functions
– ident: ref27
  doi: 10.1109/INFOCOM41043.2020.9155347
– volume: 10
  start-page: 569
  year: 2009
  ident: ref28
  article-title: Online learning with sample path constraints
  publication-title: J. Mach. Learn. Res.
– volume: 13
  start-page: 2503
  year: 2012
  ident: ref12
  article-title: Trading regret for efficiency: Online convex optimization with long term constraints
  publication-title: J. Mach. Learn. Res.
– ident: ref30
  doi: 10.1109/TCNS.2015.2505149
– volume: 21
  start-page: 1
  year: 2020
  ident: ref14
  article-title: A low complexity algorithm with O√T) regret and O(1) constraint violations for online convex optimization with long term constraints
  publication-title: J. Mach. Learn. Res.
– ident: ref35
  doi: 10.1561/1300000001
– ident: ref40
  doi: 10.1109/INFOCOM.2018.8486305
– ident: ref24
  doi: 10.1007/978-3-031-79995-2
– ident: ref2
  doi: 10.1561/2200000018
– ident: ref26
  doi: 10.1109/TWC.2017.2690442
– start-page: 2933
  volume-title: Proc. Int. Conf. Mach. Learn. (ICML)
  ident: ref32
  article-title: Distributed online optimization over a heterogeneous network with any-batch mirror descent
– ident: ref29
  doi: 10.1109/TAC.2011.2161027
– ident: ref41
  doi: 10.1109/TWC.2013.072513.121842
– ident: ref39
  doi: 10.1109/71.993206
SSID ssj0013026
Score 2.4547582
Snippet We consider online convex optimization (OCO) with multi-slot feedback delay. An agent selects a sequence of online decisions to minimize the accumulation of...
SourceID proquest
crossref
ieee
SourceType Aggregation Database
Enrichment Source
Index Database
Publisher
StartPage 147
SubjectTerms Algorithms
Benchmark testing
Benchmarks
Computational geometry
constraint violation
Constraints
Convexity
Decision theory
Delay
Delays
dynamic regret
Feedback
Heuristic algorithms
long-term constraint
Mathematical analysis
multi-slot delay
Online convex optimization
online network resource allocation
Optimization
Regularization
Resource allocation
Resource management
Upper bounds
Wireless communication
Title Delay-Tolerant OCO With Long-Term Constraints: Algorithm and Its Application to Network Resource Allocation
URI https://ieeexplore.ieee.org/document/9831158
https://www.proquest.com/docview/2776794830
Volume 31
WOSCitedRecordID wos000829067600001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVIEE
  databaseName: IEEE Electronic Library (IEL)
  customDbUrl:
  eissn: 1558-2566
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0013026
  issn: 1063-6692
  databaseCode: RIE
  dateStart: 19930101
  isFulltext: true
  titleUrlDefault: https://ieeexplore.ieee.org/
  providerName: IEEE
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NS8QwEB1UPOjBb3F1lRw8idG26aaJt2VVFGT1UD9uJUkTFddWdqvgvzdJu6uiCN4Kk4G2r81kMpP3AHbzXNhlQxxiFcsOjqmhWBgaYCFNrjWLiPBkOjcXSb_P7u741RTsT87CaK1985k-cJe-lp-X6tVtlR1y5rhh2DRMJwmtz2p9VgwCL61mMxyCKeVRU8EMA36Y9k9SmwlGkU1QmWNs-xaDvKjKj5nYh5fTxf_d2BIsNMtI1K1xX4YpXazA_BdywVV4OtYD8Y7TcqBtPKrQZe8S3T5WD-iiLO5xamdk5NQ6vUZENTpC3cF9ObT2ZySKHJ1XI9T9LG6jqkT9umMcjXf8rYeLhM68BtenJ2nvDDfSCljZ-F7Z9NNwmQhiQqpCE6vcGEqZa4ySdolAQ0mYkTokWhGuSGCUjoxJBGOKsIQLQ9ZhpigLvQEosCabVMXSIh5TJaXOqelEggrTSXIVtSAYv-xMNbzj7tEGmc8_Ap45fDKHT9bg04K9ictLTbrx1-BVB8hkYINFC9pjRLPmtxxlkeMu4jEjwebvXlsw5_Tk67bsNsxUw1e9DbPqrXocDXf8F_cB0t3VjA
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3BbtQwEB2VFgl6oEBBbCngQ08IU8f2Oja3VWnVim3KIUBvke3YpWJJ0G5aqX-P7WS3IBASt0jjkZK8xOPxjN8D2KtrHZYNPMOWmzHmwgusvSBYG187JynTiUzn8zQvCnl-rj6uwZvVWRjnXGo-c2_jZarl1629iltl-0pGbhh5BzbGnFPSn9a6rRmQJK4WchyGhVB0qGFmRO2XxWEZckFKQ4oqI2fbb1Eoyar8MRenAHO09X-39hAeDAtJNOmRfwRrrnkMm7_QC27Dt_dupm9w2c5ciEgdOjs4Q18uu69o2jYXuAxzMop6nUklolu8Q5PZRTsP9u9INzU66RZoclveRl2Lir5nHC33_INHjIXR_AQ-HR2WB8d4EFfANkT4LiSgXplcM58Jm3lua--FkLE1yoRFgsgMk964jDnLlGXEW0e9z7WUlslcac-ewnrTNu4ZIBJMIa3iJmDOhTXG1cKPqRbaj_Pa0hGQ5cuu7MA8Hh9tVqUMhKgq4lNFfKoBnxG8Xrn86Gk3_jV4OwKyGjhgMYLdJaLV8GMuKhrZixSXjOz83esV3DsuT6fV9KT48BzuR3X5vkl7F9a7-ZV7AXftdXe5mL9MX99PFQ3Y0w
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Delay-Tolerant+OCO+With+Long-Term+Constraints%3A+Algorithm+and+Its+Application+to+Network+Resource+Allocation&rft.jtitle=IEEE%2FACM+transactions+on+networking&rft.au=Wang%2C+Juncheng&rft.au=Dong%2C+Min&rft.au=Liang%2C+Ben&rft.au=Boudreau%2C+Gary&rft.date=2023-02-01&rft.pub=IEEE&rft.issn=1063-6692&rft.volume=31&rft.issue=1&rft.spage=147&rft.epage=163&rft_id=info:doi/10.1109%2FTNET.2022.3188285&rft.externalDocID=9831158
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1063-6692&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1063-6692&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1063-6692&client=summon