Markov \alpha-Potential Games

We propose a new framework of Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential games to study Markov games. We show that any Markov game with finite-state and finite-action is a Markov <inline-formula><tex-math...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:IEEE transactions on automatic control s. 1 - 16
Hlavní autori: Guo, Xin, Li, Xinyu, Maheshwari, Chinmay, Sastry, Shankar, Wu, Manxi
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: IEEE 2025
Predmet:
Approximation algorithms
Equilibrium approximation algorithms
Games
Heuristic algorithms
Linear programming
Markov games
Markov potential games
Measurement
Multi-agent reinforcement learning
Nash equilibrium
Postal services
Regret analysis
Topology
Training
Upper bound
ISSN:0018-9286, 1558-2523
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:We propose a new framework of Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential games to study Markov games. We show that any Markov game with finite-state and finite-action is a Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential game, and establish the existence of an associated <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential function. Any optimizer of an <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential function is shown to be an <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-stationary Nash equilibrium. We study two important classes of practically significant Markov games, Markov congestion games and the perturbed Markov team games, via the framework of Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential games, with explicit characterization of an upper bound for <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula> and its relation to game parameters. Additionally, we provide a semi-infinite linear programming based formulation to obtain an upper bound for <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula> for any Markov game. Furthermore, we study two equilibrium approximation algorithms, namely the projected gradient-ascent algorithm and the sequential maximum improvement algorithm, along with their Nash regret analysis, and corroborate the results with numerical experiments.
ISSN:0018-9286
1558-2523
DOI:10.1109/TAC.2025.3589416