A Strongly Polynomial Algorithm for Controlled Queues

We consider the problem of computing optimal policies of finite-state finite-action Markov decision processes (MDPs). A reduction to a continuum of constrained MDPs (CMDPs) is presented such that the optimal policies for these CMDPs constitute a path in a graph defined over the deterministic policie...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Mathematics of operations research Jg. 34; H. 4; S. 992 - 1007
Hauptverfasser: Zadorojniy, Alexander, Even, Guy, Shwartz, Adam
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Linthicum INFORMS 01.11.2009
Institute for Operations Research and the Management Sciences
Schlagworte:
1 queue
Algorithms
Analysis
Average cost
constrained Markov decision process
controlled queues
Determinism
Linear programming
M/M
Markov analysis
Markov decision process
Markov processes
Mathematical vectors
Matrices
Optimal policy
optimization
Polynomials
Queuing theory
Studies
Uniqueness
United States
ISSN:0364-765X, 1526-5471
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:We consider the problem of computing optimal policies of finite-state finite-action Markov decision processes (MDPs). A reduction to a continuum of constrained MDPs (CMDPs) is presented such that the optimal policies for these CMDPs constitute a path in a graph defined over the deterministic policies. This path contains, in particular, an optimal policy of the original MDP. We present an algorithm based on this new approach that finds this path, and thus an optimal policy. In the general case, this path might be exponentially long in the number of states and actions. We prove that the length of this path is polynomial if the MDP satisfies a coupling property. Thus we obtain a strongly polynomial algorithm for MDP s that satisfies the coupling property. We prove that discrete time versions of controlled M/M /1 queues induce MDP s that satisfy the coupling property. The only previously known polynomial algorithm for controlled M/M /1 queues in the expected average cost model is based on linear programming (and is not known to be strongly polynomial). Our algorithm works both for the discounted and expected average cost models, and the running time does not depend on the discount factor.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0364-765X
1526-5471
DOI:10.1287/moor.1090.0415