Markov \alpha-Potential Games

We propose a new framework of Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential games to study Markov games. We show that any Markov game with finite-state and finite-action is a Markov <inline-formula><tex-math...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on automatic control pp. 1 - 16
Main Authors: Guo, Xin, Li, Xinyu, Maheshwari, Chinmay, Sastry, Shankar, Wu, Manxi
Format: Journal Article
Language:English
Published: IEEE 2025
Subjects:
Approximation algorithms
Equilibrium approximation algorithms
Games
Heuristic algorithms
Linear programming
Markov games
Markov potential games
Measurement
Multi-agent reinforcement learning
Nash equilibrium
Postal services
Regret analysis
Topology
Training
Upper bound
ISSN:0018-9286, 1558-2523
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract We propose a new framework of Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential games to study Markov games. We show that any Markov game with finite-state and finite-action is a Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential game, and establish the existence of an associated <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential function. Any optimizer of an <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential function is shown to be an <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-stationary Nash equilibrium. We study two important classes of practically significant Markov games, Markov congestion games and the perturbed Markov team games, via the framework of Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential games, with explicit characterization of an upper bound for <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula> and its relation to game parameters. Additionally, we provide a semi-infinite linear programming based formulation to obtain an upper bound for <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula> for any Markov game. Furthermore, we study two equilibrium approximation algorithms, namely the projected gradient-ascent algorithm and the sequential maximum improvement algorithm, along with their Nash regret analysis, and corroborate the results with numerical experiments.
AbstractList We propose a new framework of Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential games to study Markov games. We show that any Markov game with finite-state and finite-action is a Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential game, and establish the existence of an associated <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential function. Any optimizer of an <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential function is shown to be an <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-stationary Nash equilibrium. We study two important classes of practically significant Markov games, Markov congestion games and the perturbed Markov team games, via the framework of Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential games, with explicit characterization of an upper bound for <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula> and its relation to game parameters. Additionally, we provide a semi-infinite linear programming based formulation to obtain an upper bound for <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula> for any Markov game. Furthermore, we study two equilibrium approximation algorithms, namely the projected gradient-ascent algorithm and the sequential maximum improvement algorithm, along with their Nash regret analysis, and corroborate the results with numerical experiments.
Author Guo, Xin
Wu, Manxi
Li, Xinyu
Maheshwari, Chinmay
Sastry, Shankar
Author_xml – sequence: 1
  givenname: Xin
  surname: Guo
  fullname: Guo, Xin
  email: xinguo@berkeley.edu
  organization: Department of Industrial Engineering and Operations Research, University of California, Berkeley, Berkeley, CA, USA
– sequence: 2
  givenname: Xinyu
  surname: Li
  fullname: Li, Xinyu
  email: xinyu_li@berkeley.edu
  organization: Department of Industrial Engineering and Operations Research, University of California, Berkeley, Berkeley, CA, USA
– sequence: 3
  givenname: Chinmay
  surname: Maheshwari
  fullname: Maheshwari, Chinmay
  email: chinmay_maheshwari@jhu.edu
  organization: Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA
– sequence: 4
  givenname: Shankar
  surname: Sastry
  fullname: Sastry, Shankar
  email: sastry@eecs.berkeley.edu
  organization: Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, CA, USA
– sequence: 5
  givenname: Manxi
  surname: Wu
  fullname: Wu, Manxi
  email: manxiwu@berkeley.edu
  organization: Department of Civil and Environmental Engineering, University of California, Berkeley, Berkeley, CA, USA
BookMark eNp9j8FKw0AQhhepYFq9e1DoC2yc2c0kk2Mp2goVPdSbEKbbDUbTpCSL4Nub0oLgQeYwzMD3_3xjNWraxit1jRAjQn63ns1jA4ZiS5wnmJ6pCIlYGzJ2pCIAZJ0bTi_UuO8_hjNNEozU7ZN0n-3X9E3q_bvolzb4JlRSTxey8_2lOi-l7v3VaU_U68P9er7Uq-fF43y20g4Zg059CQZI8oyIhCXfEJJl4zlDuymHcbSlxGWIuQPcskuIcfhZUxIK24mCY67r2r7vfFnsu2on3XeBUBz0ikGvOOgVJ70BSf8grgoSqrYJnVT1f-DNEay89789CAyG0f4AAp5cxA
CODEN IETAA9
CitedBy_id crossref_primary_10_1137_24M1707316
ContentType Journal Article
DBID 97E
RIA
RIE
AAYXX
CITATION
DOI 10.1109/TAC.2025.3589416
DatabaseName IEEE All-Society Periodicals Package (ASPP) 2005–Present
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE/IET Electronic Library
CrossRef
DatabaseTitle CrossRef
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Xplore
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
EISSN 1558-2523
EndPage 16
ExternalDocumentID 10_1109_TAC_2025_3589416
11080281
Genre orig-research
GroupedDBID -~X
.DC
0R~
29I
4.4
5GY
6IK
97E
AAJGR
AARMG
AASAJ
AAWTH
ABAZT
ABQJQ
ABVLG
ACGFO
ACGFS
ACIWK
ACNCT
AENEX
AGQYO
AHBIQ
AKJIK
AKQYR
ALMA_UNASSIGNED_HOLDINGS
ASUFR
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CS3
DU5
EBS
F5P
HZ~
IFIPE
IPLJI
JAVBF
LAI
M43
MS~
O9-
OCL
P2P
RIA
RIE
RNS
TAE
TN5
~02
3EH
5VS
AAYXX
AETIX
AGSQL
AI.
AIBXA
ALLEH
CITATION
EJD
H~9
IAAWW
IBMZZ
ICLAB
IDIHD
IFJZH
VH1
VJK
ID FETCH-LOGICAL-c181t-6ef0205a97555a8a9b515382e8713bfbfbc5d54c7119c01d8c4581c5d32f51a83
IEDL.DBID RIE
ISSN 0018-9286
IngestDate Sat Nov 29 07:41:54 EST 2025
Tue Nov 18 22:27:42 EST 2025
Wed Jul 23 05:50:25 EDT 2025
IsPeerReviewed true
IsScholarly true
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
https://doi.org/10.15223/policy-029
https://doi.org/10.15223/policy-037
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c181t-6ef0205a97555a8a9b515382e8713bfbfbc5d54c7119c01d8c4581c5d32f51a83
PageCount 16
ParticipantIDs ieee_primary_11080281
crossref_primary_10_1109_TAC_2025_3589416
crossref_citationtrail_10_1109_TAC_2025_3589416
PublicationCentury 2000
PublicationDate 2025-00-00
PublicationDateYYYYMMDD 2025-01-01
PublicationDate_xml – year: 2025
  text: 2025-00-00
PublicationDecade 2020
PublicationTitle IEEE transactions on automatic control
PublicationTitleAbbrev TAC
PublicationYear 2025
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0016441
Score 2.468364
Snippet We propose a new framework of Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential games to study Markov games. We...
SourceID crossref
ieee
SourceType Enrichment Source
Index Database
Publisher
StartPage 1
SubjectTerms Approximation algorithms
Equilibrium approximation algorithms
Games
Heuristic algorithms
Linear programming
Markov games
Markov potential games
Measurement
Multi-agent reinforcement learning
Nash equilibrium
Postal services
Regret analysis
Topology
Training
Upper bound
Title Markov \alpha-Potential Games
URI https://ieeexplore.ieee.org/document/11080281
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVIEE
  databaseName: IEEE Xplore
  customDbUrl:
  eissn: 1558-2523
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0016441
  issn: 0018-9286
  databaseCode: RIE
  dateStart: 19630101
  isFulltext: true
  titleUrlDefault: https://ieeexplore.ieee.org/
  providerName: IEEE
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NS8MwFH_o8KAHPydOnezgxUO2Nh9NchzD6WnsMGEHobykGQiyyuz295ukde6iIL2UkAflF_o-8j5-APeZShmawhGFhSTc-xwEFS9IgoyhygqkhkeyCTmZqPlcT5tm9dgL45yLxWeuH15jLr8o7TpclQ1Cybq3hz7Y2Zcyq5u1timDYNhrtev_YKq2OclED2bDkY8EqegzoTQP1OY7NmiHVCXalPHJP7_mFI4b57E3rE_7DPbc8hyOdkYKXkA3NN-Um95rbKIl07IK5UBe6CmUw7bhZfw4Gz2ThgKBWG96K5K5hffnBGophECF2oigoqjzcQ4zC_9YUQhuZZpqm6SFslyo1K8xuhApKnYJrWW5dFfQSyhmRicWmaNeOzJjlLFMcaclShS8A4NvUHLbzAcPNBXveYwTEp17GPMAY97A2IGHrcRHPRvjj73tgODPvga861_Wb-AwiNeXHbfQqlZr14UDu6nePld38eS_AHi7qE8
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3LS8MwGP-QKagHnxOnTnfw4iFb2yRtchzDOXGOHSbsIJS8BoKsMrv9_SZpnLsoSC8lJKX80n6PfI8fwG3KYiykNogJnSFibQ4kGNEoEhgLlmqRSOLJJrLRiE2nfByK1X0tjDHGJ5-Ztrv1sXxdqKU7Kuu4lHWrD62zs-2os0K51jpo4FR7JXjtP5ywdVQy4p1Jt2d9wYS2MWWcOHLzDS20QavitUr_8J_vcwQHwXxsdav9PoYtMz-B_Y2mgqfQdOU3xar16sto0bgoXUKQXfTgEmLr8NK_n_QGKJAgIGWVb4lSM7MWHRU8o5QKJrikTkglxno6WM7spaimRGVxzFUUa6YIZbEdw8mMxoLhM6jNi7k5h1aUiFTySAlsEisfsZRMKsyI4ZnIBCUN6HyDkqvQIdwRVbzn3lOIeG5hzB2MeYCxAXfrFR9Vd4w_5tYdgj_zAngXv4zfwO5g8jzMh4-jp0vYc4-qjj6uoFYulqYJO2pVvn0urv1X8AVzxauY
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Markov+%24%5Calpha%24-Potential+Games&rft.jtitle=IEEE+transactions+on+automatic+control&rft.au=Guo%2C+Xin&rft.au=Li%2C+Xinyu&rft.au=Maheshwari%2C+Chinmay&rft.au=Sastry%2C+Shankar&rft.date=2025&rft.issn=0018-9286&rft.eissn=1558-2523&rft.spage=1&rft.epage=16&rft_id=info:doi/10.1109%2FTAC.2025.3589416&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TAC_2025_3589416
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9286&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9286&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9286&client=summon