Comparison of algorithms for simple stochastic games

Simple stochastic games are turn-based 2½-player zero-sum graph games with a reachability objective. The problem is to compute the winning probabilities as well as the optimal strategies of both players. In this paper, we compare the three known classes of algorithms – value iteration, strategy iter...

Full description

Saved in:
Bibliographic Details
Published in:Information and computation Vol. 289; p. 104885
Main Authors: Křetínský, Jan, Ramneantu, Emanuel, Slivinskiy, Alexander, Weininger, Maximilian
Format: Journal Article
Language:English
Published: Elsevier Inc 01.11.2022
Subjects:
Algorithms
Formal methods
Probabilistic verification
Quadratic programming
Stochastic games
Strategy iteration
Value iteration
Stochastic games
Strategy iteration
Algorithms
Probabilistic verification
Value iteration
Formal methods
Quadratic programming
ISSN:0890-5401, 1090-2651
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Simple stochastic games are turn-based 2½-player zero-sum graph games with a reachability objective. The problem is to compute the winning probabilities as well as the optimal strategies of both players. In this paper, we compare the three known classes of algorithms – value iteration, strategy iteration and quadratic programming – both theoretically and practically. Further, we suggest several improvements for all algorithms, including the first approach based on quadratic programming that avoids transforming the stochastic game to a stopping one. Our extensive experiments show that these improvements can lead to significant speed-ups. We implemented all algorithms in PRISM-games 3.0, thereby providing the first implementation of quadratic programming for solving simple stochastic games.
ISSN:0890-5401
1090-2651
DOI:10.1016/j.ic.2022.104885