Markov \alpha-Potential Games
We propose a new framework of Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential games to study Markov games. We show that any Markov game with finite-state and finite-action is a Markov <inline-formula><tex-math...
Gespeichert in:
| Veröffentlicht in: | IEEE transactions on automatic control S. 1 - 16 |
|---|---|
| Hauptverfasser: | , , , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
IEEE
2025
|
| Schlagworte: | |
| ISSN: | 0018-9286, 1558-2523 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | We propose a new framework of Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential games to study Markov games. We show that any Markov game with finite-state and finite-action is a Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential game, and establish the existence of an associated <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential function. Any optimizer of an <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential function is shown to be an <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-stationary Nash equilibrium. We study two important classes of practically significant Markov games, Markov congestion games and the perturbed Markov team games, via the framework of Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential games, with explicit characterization of an upper bound for <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula> and its relation to game parameters. Additionally, we provide a semi-infinite linear programming based formulation to obtain an upper bound for <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula> for any Markov game. Furthermore, we study two equilibrium approximation algorithms, namely the projected gradient-ascent algorithm and the sequential maximum improvement algorithm, along with their Nash regret analysis, and corroborate the results with numerical experiments. |
|---|---|
| AbstractList | We propose a new framework of Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential games to study Markov games. We show that any Markov game with finite-state and finite-action is a Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential game, and establish the existence of an associated <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential function. Any optimizer of an <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential function is shown to be an <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-stationary Nash equilibrium. We study two important classes of practically significant Markov games, Markov congestion games and the perturbed Markov team games, via the framework of Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential games, with explicit characterization of an upper bound for <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula> and its relation to game parameters. Additionally, we provide a semi-infinite linear programming based formulation to obtain an upper bound for <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula> for any Markov game. Furthermore, we study two equilibrium approximation algorithms, namely the projected gradient-ascent algorithm and the sequential maximum improvement algorithm, along with their Nash regret analysis, and corroborate the results with numerical experiments. |
| Author | Guo, Xin Wu, Manxi Li, Xinyu Maheshwari, Chinmay Sastry, Shankar |
| Author_xml | – sequence: 1 givenname: Xin surname: Guo fullname: Guo, Xin email: xinguo@berkeley.edu organization: Department of Industrial Engineering and Operations Research, University of California, Berkeley, Berkeley, CA, USA – sequence: 2 givenname: Xinyu surname: Li fullname: Li, Xinyu email: xinyu_li@berkeley.edu organization: Department of Industrial Engineering and Operations Research, University of California, Berkeley, Berkeley, CA, USA – sequence: 3 givenname: Chinmay surname: Maheshwari fullname: Maheshwari, Chinmay email: chinmay_maheshwari@jhu.edu organization: Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA – sequence: 4 givenname: Shankar surname: Sastry fullname: Sastry, Shankar email: sastry@eecs.berkeley.edu organization: Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, CA, USA – sequence: 5 givenname: Manxi surname: Wu fullname: Wu, Manxi email: manxiwu@berkeley.edu organization: Department of Civil and Environmental Engineering, University of California, Berkeley, Berkeley, CA, USA |
| BookMark | eNp9j8FKw0AQhhepYFq9e1DoC2yc2c0kk2Mp2goVPdSbEKbbDUbTpCSL4Nub0oLgQeYwzMD3_3xjNWraxit1jRAjQn63ns1jA4ZiS5wnmJ6pCIlYGzJ2pCIAZJ0bTi_UuO8_hjNNEozU7ZN0n-3X9E3q_bvolzb4JlRSTxey8_2lOi-l7v3VaU_U68P9er7Uq-fF43y20g4Zg059CQZI8oyIhCXfEJJl4zlDuymHcbSlxGWIuQPcskuIcfhZUxIK24mCY67r2r7vfFnsu2on3XeBUBz0ikGvOOgVJ70BSf8grgoSqrYJnVT1f-DNEay89789CAyG0f4AAp5cxA |
| CODEN | IETAA9 |
| CitedBy_id | crossref_primary_10_1137_24M1707316 |
| ContentType | Journal Article |
| DBID | 97E RIA RIE AAYXX CITATION |
| DOI | 10.1109/TAC.2025.3589416 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1558-2523 |
| EndPage | 16 |
| ExternalDocumentID | 10_1109_TAC_2025_3589416 11080281 |
| Genre | orig-research |
| GroupedDBID | -~X .DC 0R~ 29I 4.4 5GY 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK ACNCT AENEX AGQYO AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS F5P HZ~ IFIPE IPLJI JAVBF LAI M43 MS~ O9- OCL P2P RIA RIE RNS TAE TN5 ~02 3EH 5VS AAYXX AETIX AGSQL AI. AIBXA ALLEH CITATION EJD H~9 IAAWW IBMZZ ICLAB IDIHD IFJZH VH1 VJK |
| ID | FETCH-LOGICAL-c181t-6ef0205a97555a8a9b515382e8713bfbfbc5d54c7119c01d8c4581c5d32f51a83 |
| IEDL.DBID | RIE |
| ISSN | 0018-9286 |
| IngestDate | Sat Nov 29 07:41:54 EST 2025 Tue Nov 18 22:27:42 EST 2025 Wed Jul 23 05:50:25 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c181t-6ef0205a97555a8a9b515382e8713bfbfbc5d54c7119c01d8c4581c5d32f51a83 |
| PageCount | 16 |
| ParticipantIDs | ieee_primary_11080281 crossref_primary_10_1109_TAC_2025_3589416 crossref_citationtrail_10_1109_TAC_2025_3589416 |
| PublicationCentury | 2000 |
| PublicationDate | 2025-00-00 |
| PublicationDateYYYYMMDD | 2025-01-01 |
| PublicationDate_xml | – year: 2025 text: 2025-00-00 |
| PublicationDecade | 2020 |
| PublicationTitle | IEEE transactions on automatic control |
| PublicationTitleAbbrev | TAC |
| PublicationYear | 2025 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0016441 |
| Score | 2.468441 |
| Snippet | We propose a new framework of Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential games to study Markov games. We... |
| SourceID | crossref ieee |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 1 |
| SubjectTerms | Approximation algorithms Equilibrium approximation algorithms Games Heuristic algorithms Linear programming Markov games Markov potential games Measurement Multi-agent reinforcement learning Nash equilibrium Postal services Regret analysis Topology Training Upper bound |
| Title | Markov \alpha-Potential Games |
| URI | https://ieeexplore.ieee.org/document/11080281 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-2523 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0016441 issn: 0018-9286 databaseCode: RIE dateStart: 19630101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEB60eNCDz4pVKz148ZB2X9kkx1KsHqT0UKUHYclmExCkK3Xb3-9Mdq29KMheQpiB5QvJZDLzzQDc5iIXQhSGmTQNWOJsynJNZHcuDdc6KmLfzuflSUwmcj5X04as7rkw1lqffGb7NPSx_KI0K3oqG1DKOtpDdHZ2hUhrstYmZECGvT52cQdHchOTDNRgNhyhJxjxfsylSqi1-ZYN2mqq4m3K-Oiff3MMh83lsTesV_sEduziFA62SgqeQZfIN-W69-pJtGxaVpQOhEoPlA7bhufx_Wz0yJoWCMyg6a1Yah3e57hWgnOupVY5pyMqsujnxLnDz_CCJ0aEoTJBWEiTcBniXBw5HmoZn0NrUS7sBfSkj9A55yiSKI3TGveqjlE0zVHSdWDwDUpmmvrg1KbiPfN-QqAyhDEjGLMGxg7cbTQ-6toYf8i2CcEfuQa8y1_mr2Cf1OvHjmtoVcuV7cKeWVdvn8sbv_JftCSodA |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEB6kCurBZ8Wq1R68eEi7r2ySYynWirX0UKUHYclmEyhIV-q2v99Mdq29KMheQpiE5dtNJpOZbwbgNmUpYyxTRMWxRyKjY5JKJLtTrqiUQRa6cj6vQzYa8elUjCuyuuPCaK1d8JluY9P58rNcLfGqrIMh61YfWmNnm0ZR4JV0rbXTAFV7ufHaNRzwtVfSE51Jt2dtwYC2Q8pFhMXNN7TQRlkVp1X6h_98nyM4qI6PrW75vY9hS89PYH8jqeApNJF-k69ab45GS8Z5gQFBdtADBsTW4aV_P-kNSFUEgSirfAsSa2NPdFQKRimVXIqU4iYVaGvphKmxj6IZjRTzfaE8P-Mqoty3fWFgqC95eAa1eT7X59DizkdnjEFfIldGSrtaZWhF49RKmgZ0vkFJVJUhHAtVvCfOUvBEYmFMEMakgrEBd-sRH2V2jD9k64jgj1wF3sUv_TewO5g8D5Ph4-jpEvZwqvLq4wpqxWKpm7CjVsXsc3Ht_oIvrPmruw |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Markov+%5Calpha-Potential+Games&rft.jtitle=IEEE+transactions+on+automatic+control&rft.au=Guo%2C+Xin&rft.au=Li%2C+Xinyu&rft.au=Maheshwari%2C+Chinmay&rft.au=Sastry%2C+Shankar&rft.date=2025&rft.pub=IEEE&rft.issn=0018-9286&rft.spage=1&rft.epage=16&rft_id=info:doi/10.1109%2FTAC.2025.3589416&rft.externalDocID=11080281 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9286&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9286&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9286&client=summon |