Markov \alpha-Potential Games

We propose a new framework of Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential games to study Markov games. We show that any Markov game with finite-state and finite-action is a Markov <inline-formula><tex-math...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on automatic control pp. 1 - 16
Main Authors: Guo, Xin, Li, Xinyu, Maheshwari, Chinmay, Sastry, Shankar, Wu, Manxi
Format: Journal Article
Language:English
Published: IEEE 2025
Subjects:
Approximation algorithms
Equilibrium approximation algorithms
Games
Heuristic algorithms
Linear programming
Markov games
Markov potential games
Measurement
Multi-agent reinforcement learning
Nash equilibrium
Postal services
Regret analysis
Topology
Training
Upper bound
ISSN:0018-9286, 1558-2523
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We propose a new framework of Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential games to study Markov games. We show that any Markov game with finite-state and finite-action is a Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential game, and establish the existence of an associated <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential function. Any optimizer of an <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential function is shown to be an <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-stationary Nash equilibrium. We study two important classes of practically significant Markov games, Markov congestion games and the perturbed Markov team games, via the framework of Markov <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula>-potential games, with explicit characterization of an upper bound for <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula> and its relation to game parameters. Additionally, we provide a semi-infinite linear programming based formulation to obtain an upper bound for <inline-formula><tex-math notation="LaTeX">\alpha</tex-math></inline-formula> for any Markov game. Furthermore, we study two equilibrium approximation algorithms, namely the projected gradient-ascent algorithm and the sequential maximum improvement algorithm, along with their Nash regret analysis, and corroborate the results with numerical experiments.
ISSN:0018-9286
1558-2523
DOI:10.1109/TAC.2025.3589416