Constrained Online Convex Optimization With Feedback Delays

In this article, we study constrained online convex optimization (OCO) in the presence of feedback delays, where a decision maker chooses sequential actions without knowing the loss functions and constraint functions a priori . The loss/constraint functions vary with time and their feedback informat...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:IEEE transactions on automatic control Ročník 66; číslo 11; s. 5049 - 5064
Hlavní autori: Cao, Xuanyu, Zhang, Junshan, Poor, H. Vincent
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: New York IEEE 01.11.2021
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:
Algorithms
Bandit feedback
Benchmark testing
Computational geometry
constrained optimization
Constraints
Convex analysis
Convexity
Decision making
Decision theory
Delays
Feedback
feedback delay
function feedback
Functionals
online convex optimization (OCO)
Optimization
Saddle points
Time factors
ISSN:0018-9286, 1558-2523
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Abstract In this article, we study constrained online convex optimization (OCO) in the presence of feedback delays, where a decision maker chooses sequential actions without knowing the loss functions and constraint functions a priori . The loss/constraint functions vary with time and their feedback information is revealed to the decision maker with delays, which arise in many applications. We first consider the scenario of delayed function feedback, in which the complete information of the loss/constraint functions is revealed to the decision maker with delays. We develop a modified online saddle point algorithm suitable for constrained OCO with feedback delays. Sublinear regret and sublinear constraint violation bounds are established for the algorithm in terms of the delays. In practice, the complete information (functional forms) of the loss/constraint functions may not be revealed to the decision maker. Thus, we further examine the scenario of delayed bandit feedback, where only the values of the loss/constraint functions at two random points close to the chosen action are revealed to the decision maker with delays. A delayed version of the bandit online saddle point algorithm is proposed by utilizing stochastic gradient estimates of the loss/constraint functions based on delayed bandit feedback. We also establish sublinear regret and sublinear constraint violation bounds for this bandit optimization algorithm in terms of the delays. Finally, numerical results for online quadratically constrained quadratic programs are presented to corroborate the efficacy of the proposed algorithms.
AbstractList In this article, we study constrained online convex optimization (OCO) in the presence of feedback delays, where a decision maker chooses sequential actions without knowing the loss functions and constraint functions a priori . The loss/constraint functions vary with time and their feedback information is revealed to the decision maker with delays, which arise in many applications. We first consider the scenario of delayed function feedback, in which the complete information of the loss/constraint functions is revealed to the decision maker with delays. We develop a modified online saddle point algorithm suitable for constrained OCO with feedback delays. Sublinear regret and sublinear constraint violation bounds are established for the algorithm in terms of the delays. In practice, the complete information (functional forms) of the loss/constraint functions may not be revealed to the decision maker. Thus, we further examine the scenario of delayed bandit feedback, where only the values of the loss/constraint functions at two random points close to the chosen action are revealed to the decision maker with delays. A delayed version of the bandit online saddle point algorithm is proposed by utilizing stochastic gradient estimates of the loss/constraint functions based on delayed bandit feedback. We also establish sublinear regret and sublinear constraint violation bounds for this bandit optimization algorithm in terms of the delays. Finally, numerical results for online quadratically constrained quadratic programs are presented to corroborate the efficacy of the proposed algorithms.
Author Zhang, Junshan
Cao, Xuanyu
Poor, H. Vincent
Author_xml – sequence: 1
  givenname: Xuanyu
  orcidid: 0000-0003-0190-4362
  surname: Cao
  fullname: Cao, Xuanyu
  email: xyc@illinois.edu
  organization: Coordinated Science Laboratory, University of Illinois at Urbana-Champaign, Urbana, IL, USA
– sequence: 2
  givenname: Junshan
  orcidid: 0000-0002-3840-1753
  surname: Zhang
  fullname: Zhang, Junshan
  email: junshan.zhang@asu.edu
  organization: School of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, AZ, USA
– sequence: 3
  givenname: H. Vincent
  orcidid: 0000-0002-2062-131X
  surname: Poor
  fullname: Poor, H. Vincent
  email: poor@princeton.edu
  organization: Department of Electrical Engineering, Princeton University, Princeton, NJ, USA
BookMark eNp9UE1LAzEUDFLBtnoXvCx43pq8fOwGT6VaFQq9VDyGbPoWU7e7dbMV6683tcWDB0_DPGbeezMD0qubGgm5ZHTEGNU3i_FkBBToiFNOM8FPSJ9JmacggfdIn1KWpxpydUYGIawiVUKwPrmdNHXoWutrXCbzuoqYxNEHfibzTefX_st2vqmTF9-9JlPEZWHdW3KHld2Fc3Ja2irgxRGH5Hl6v5g8prP5w9NkPEsdaNalTOpC53bphMgkAo0MOChkzkIBNJc5aoDCZqXkzgmni0xnWYkCVaGAaT4k14e9m7Z532LozKrZtnU8aUDmSoHiXEWVOqhc24TQYmmc736e38erDKNmX5SJRZl9UeZYVDTSP8ZN69e23f1nuTpYPCL-ymMKkFLwb5Uic4c
CODEN IETAA9
CitedBy_id crossref_primary_10_1109_TAC_2024_3468403
crossref_primary_10_1016_j_neucom_2022_05_024
crossref_primary_10_1007_s11768_021_00041_3
crossref_primary_10_1109_TSMC_2024_3496564
crossref_primary_10_3390_technologies13080327
crossref_primary_10_1109_JSAC_2023_3242700
crossref_primary_10_1109_LCSYS_2022_3155067
crossref_primary_10_1007_s40305_023_00496_y
crossref_primary_10_1287_ijoc_2022_1228
crossref_primary_10_1109_TCNS_2024_3432250
crossref_primary_10_1109_TNSE_2024_3409755
crossref_primary_10_1109_TAC_2023_3237781
crossref_primary_10_1016_j_arcontrol_2023_100904
crossref_primary_10_1109_LWC_2022_3152737
crossref_primary_10_1109_TPWRS_2023_3282368
crossref_primary_10_1016_j_amc_2023_127865
crossref_primary_10_1080_00207721_2023_2209566
crossref_primary_10_1109_TICPS_2025_3538690
crossref_primary_10_1016_j_eswa_2025_127871
crossref_primary_10_1016_j_neucom_2025_131406
crossref_primary_10_1109_TNET_2022_3188285
Cites_doi 10.1109/TSP.2015.2449255
10.1109/ICC.2018.8422559
10.1561/2400000013
10.1109/TSP.2017.2750109
10.1109/JIOT.2018.2839563
10.1561/2200000018
10.1109/TAC.2016.2627401
10.1109/TAC.2018.2884653
ContentType Journal Article
Copyright Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021
Copyright_xml – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021
DBID 97E
RIA
RIE
AAYXX
CITATION
7SC
7SP
7TB
8FD
FR3
JQ2
L7M
L~C
L~D
DOI 10.1109/TAC.2020.3030743
DatabaseName IEEE All-Society Periodicals Package (ASPP) 2005–Present
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE Electronic Library (IEL)
CrossRef
Computer and Information Systems Abstracts
Electronics & Communications Abstracts
Mechanical & Transportation Engineering Abstracts
Technology Research Database
Engineering Research Database
ProQuest Computer Science Collection
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts – Academic
Computer and Information Systems Abstracts Professional
DatabaseTitle CrossRef
Technology Research Database
Computer and Information Systems Abstracts – Academic
Mechanical & Transportation Engineering Abstracts
Electronics & Communications Abstracts
ProQuest Computer Science Collection
Computer and Information Systems Abstracts
Engineering Research Database
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts Professional
DatabaseTitleList
Technology Research Database
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
EISSN 1558-2523
EndPage 5064
ExternalDocumentID 10_1109_TAC_2020_3030743
9222554
Genre orig-research
GrantInformation_xml – fundername: Defense Threat Reduction Agency
  grantid: HDTRA1-13-1-0029
  funderid: 10.13039/100000774
– fundername: Army Research Office
  grantid: W911NF-16-1-0448
  funderid: 10.13039/100000183
GroupedDBID -~X
.DC
0R~
29I
3EH
4.4
5GY
5VS
6IK
97E
AAJGR
AARMG
AASAJ
AAWTH
ABAZT
ABQJQ
ABVLG
ACGFO
ACGFS
ACIWK
ACNCT
AENEX
AETIX
AGQYO
AGSQL
AHBIQ
AI.
AIBXA
AKJIK
AKQYR
ALLEH
ALMA_UNASSIGNED_HOLDINGS
ASUFR
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CS3
DU5
EBS
EJD
F5P
HZ~
H~9
IAAWW
IBMZZ
ICLAB
IDIHD
IFIPE
IFJZH
IPLJI
JAVBF
LAI
M43
MS~
O9-
OCL
P2P
RIA
RIE
RNS
TAE
TN5
VH1
VJK
~02
AAYXX
CITATION
7SC
7SP
7TB
8FD
FR3
JQ2
L7M
L~C
L~D
ID FETCH-LOGICAL-c291t-159b98adc4475e209b92326e1ca2b20858e922ba7f53cc4c9b7977fe4e6b62193
IEDL.DBID RIE
ISICitedReferencesCount 29
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000711740700004&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 0018-9286
IngestDate Mon Jun 30 10:07:48 EDT 2025
Tue Nov 18 22:10:31 EST 2025
Sat Nov 29 05:41:00 EST 2025
Wed Aug 27 03:03:24 EDT 2025
IsPeerReviewed true
IsScholarly true
Issue 11
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
https://doi.org/10.15223/policy-029
https://doi.org/10.15223/policy-037
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c291t-159b98adc4475e209b92326e1ca2b20858e922ba7f53cc4c9b7977fe4e6b62193
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0000-0003-0190-4362
0000-0002-3840-1753
0000-0002-2062-131X
PQID 2586626336
PQPubID 85475
PageCount 16
ParticipantIDs crossref_citationtrail_10_1109_TAC_2020_3030743
proquest_journals_2586626336
ieee_primary_9222554
crossref_primary_10_1109_TAC_2020_3030743
PublicationCentury 2000
PublicationDate 2021-11-01
PublicationDateYYYYMMDD 2021-11-01
PublicationDate_xml – month: 11
  year: 2021
  text: 2021-11-01
  day: 01
PublicationDecade 2020
PublicationPlace New York
PublicationPlace_xml – name: New York
PublicationTitle IEEE transactions on automatic control
PublicationTitleAbbrev TAC
PublicationYear 2021
Publisher IEEE
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml – name: IEEE
– name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References cutkosky (ref3) 0
ref24
ref12
ref23
langford (ref15) 0
mahdavi (ref6) 2012; 13
ref14
joulani (ref16) 0
liakopoulos (ref22) 0
yu (ref10) 0
cesa-bianchi (ref20) 0
ref11
ref1
zinkevich (ref2) 0
neely (ref9) 2017
li (ref18) 0
ref8
quanrud (ref17) 0
ref7
shamir (ref19) 0
rivera (ref13) 2018
mannor (ref21) 2009; 10
flaxman (ref4) 0
agarwal (ref5) 0
References_xml – start-page: 28
  year: 0
  ident: ref5
  article-title: Optimal algorithms for online convex optimization with multi-point bandit feedback
– start-page: 3944
  year: 0
  ident: ref22
  article-title: Cautious regret minimization: Online optimization with long-term budget constraints
  publication-title: Proc Int Conf Mach Learn
– start-page: 1160
  year: 0
  ident: ref20
  article-title: Online learning with switching costs and other adaptive adversaries
  publication-title: Proc Adv Neural Inf Process Syst
– year: 2018
  ident: ref13
  article-title: The online saddle point problem: Applications to online convex optimization with knapsacks
– ident: ref14
  doi: 10.1109/TSP.2015.2449255
– ident: ref23
  doi: 10.1109/ICC.2018.8422559
– start-page: 993
  year: 0
  ident: ref18
  article-title: Bandit online learning with unknown delays
  publication-title: Proc 22nd Int Conf Artif Intell Statist
– start-page: 928
  year: 0
  ident: ref2
  article-title: Online convex programming and generalized infinitesimal gradient ascent
  publication-title: Proc Int Conf Mach Learn
– ident: ref1
  doi: 10.1561/2400000013
– volume: 10
  start-page: 569
  year: 2009
  ident: ref21
  article-title: Online learning with sample path constraints
  publication-title: J Mach Learn Res
– start-page: 2331
  year: 0
  ident: ref15
  article-title: Slow learners are fast
  publication-title: Proc Neural Inf Process Syst
– start-page: 1270
  year: 0
  ident: ref17
  article-title: Online learning with adversarial delays
  publication-title: Proc Neural Inf Process Syst
– ident: ref8
  doi: 10.1109/TSP.2017.2750109
– start-page: 3086
  year: 0
  ident: ref19
  article-title: Online learning with local permutations and delayed feedback
  publication-title: Proc Int Conf Mach Learn
– ident: ref11
  doi: 10.1109/JIOT.2018.2839563
– year: 2017
  ident: ref9
  article-title: Online convex optimization with time-varying constraints
– start-page: 1428
  year: 0
  ident: ref10
  article-title: Online convex optimization with stochastic constraints
  publication-title: Proc Neural Inf Process Syst
– start-page: 1453
  year: 0
  ident: ref16
  article-title: Online learning under delayed feedback
  publication-title: Proc Int Conf Mach Learn
– ident: ref24
  doi: 10.1561/2200000018
– ident: ref7
  doi: 10.1109/TAC.2016.2627401
– start-page: 748
  year: 0
  ident: ref3
  article-title: Online convex optimization with unconstrained domains and losses
  publication-title: Proc Neural Inf Process Syst
– volume: 13
  start-page: 2503
  year: 2012
  ident: ref6
  article-title: Trading regret for efficiency: Online convex optimization with long term constraints
  publication-title: J Mach Learn Res
– ident: ref12
  doi: 10.1109/TAC.2018.2884653
– start-page: 385
  year: 0
  ident: ref4
  article-title: Online convex optimization in the bandit setting: Gradient descent without a gradient
  publication-title: Proc ACM-SIAM Symp Discrete Algorithms
SSID ssj0016441
Score 2.5085428
Snippet In this article, we study constrained online convex optimization (OCO) in the presence of feedback delays, where a decision maker chooses sequential actions...
SourceID proquest
crossref
ieee
SourceType Aggregation Database
Enrichment Source
Index Database
Publisher
StartPage 5049
SubjectTerms Algorithms
Bandit feedback
Benchmark testing
Computational geometry
constrained optimization
Constraints
Convex analysis
Convexity
Decision making
Decision theory
Delays
Feedback
feedback delay
function feedback
Functionals
online convex optimization (OCO)
Optimization
Saddle points
Time factors
Title Constrained Online Convex Optimization With Feedback Delays
URI https://ieeexplore.ieee.org/document/9222554
https://www.proquest.com/docview/2586626336
Volume 66
WOSCitedRecordID wos000711740700004&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVIEE
  databaseName: IEEE Electronic Library (IEL)
  customDbUrl:
  eissn: 1558-2523
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0016441
  issn: 0018-9286
  databaseCode: RIE
  dateStart: 19630101
  isFulltext: true
  titleUrlDefault: https://ieeexplore.ieee.org/
  providerName: IEEE
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NS8MwFH_M4UEPfk1xOqUHL4J1bZLmA09jOjxNDxN3K0ma4XBusnWi_71J2hVFEby1kED6krzfe33v_R7AmcVMmXEmQ2XRIiQSj0LFaRSSOJKGShVzLn2zCdbv8-FQ3NfgoqqFMcb45DNz6R59LD-b6aX7VdYWzjlJyBqsMcaKWq0qYuBwvdC69gIjXoUkI9EedLrWEUTWP3UnmuBvEOR7qvxQxB5detv_W9cObJVWZNAptn0Xama6B5tfuAUbcOVacfoGECYLCkLRoOtSzN-DO6smXsr6y-BxnD8FPYthSurn4NpM5MdiHx56N4PubVh2Sgg1EnEeWptECS4z7ej7DIrsm7WUqIm1RMp14eTGrlFJNkqw1kQLxazdNzLEUEWtzsIHUJ_OpuYQgkQnCjMlHO8y0RQpC2DYuGipzAjDWRPaK-GluqQRdx8zSb07EYnUijt14k5LcTfhvJrxWlBo_DG24cRbjSsl24TWan_S8o4tUpRw6rh0MD36fdYxbCCXgeIrB1tQz-dLcwLr-i0fL-an_vh8AkGewA8
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3fS8MwED7mFNQHf4vTqX3wRbCuTdM2wacxHRPn9GHi3kKSZjicm2yd6H9vknZFUQTfWkggvST33fXuvgM40ZjJExJzV2i0cDEP-q4gkedi3-Mq4sInhNtmE3GnQ3o9el-Cs6IWRillk8_UuXm0sfxkLGfmV1mNGuckxAuwGGKM_Kxaq4gZGGTP9K6-wogUQUmP1rr1hnYFkfZQzZnGwTcQsl1Vfqhiiy_N9f-tbAPWcjvSqWcbvwklNdqC1S_sgttwYZpx2hYQKnEySlGnYZLM3507rShe8gpM53GQPjlNjWKCy2fnUg35x3QHHppX3UbLzXsluBJRP3W1VSIo4Yk0BH4KefpN20qR8iVHwvThJEqvUfC4HwZSYklFrC2_vsIqEpHWWsEulEfjkdoDJ5ShCGJBDfMylhESGsICZeKlPMFxkFSgNhcekzmRuPmYIbMOhUeZFjcz4ma5uCtwWsx4zUg0_hi7bcRbjMslW4HqfH9YfsumDIUkMmw6QbT_-6xjWG51b9usfd25OYAVZPJRbB1hFcrpZKYOYUm-pYPp5MgepU9uGcNW
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Constrained+Online+Convex+Optimization+With+Feedback+Delays&rft.jtitle=IEEE+transactions+on+automatic+control&rft.au=Cao%2C+Xuanyu&rft.au=Zhang%2C+Junshan&rft.au=Poor%2C+H.+Vincent&rft.date=2021-11-01&rft.pub=IEEE&rft.issn=0018-9286&rft.volume=66&rft.issue=11&rft.spage=5049&rft.epage=5064&rft_id=info:doi/10.1109%2FTAC.2020.3030743&rft.externalDocID=9222554
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9286&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9286&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9286&client=summon