A Strongly Polynomial Algorithm for Controlled Queues

We consider the problem of computing optimal policies of finite-state finite-action Markov decision processes (MDPs). A reduction to a continuum of constrained MDPs (CMDPs) is presented such that the optimal policies for these CMDPs constitute a path in a graph defined over the deterministic policie...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Mathematics of operations research Ročník 34; číslo 4; s. 992 - 1007
Hlavní autoři: Zadorojniy, Alexander, Even, Guy, Shwartz, Adam
Médium: Journal Article
Jazyk:angličtina
Vydáno: Linthicum INFORMS 01.11.2009
Institute for Operations Research and the Management Sciences
Témata:
1 queue
Algorithms
Analysis
Average cost
constrained Markov decision process
controlled queues
Determinism
Linear programming
M/M
Markov analysis
Markov decision process
Markov processes
Mathematical vectors
Matrices
Optimal policy
optimization
Polynomials
Queuing theory
Studies
Uniqueness
United States
ISSN:0364-765X, 1526-5471
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:We consider the problem of computing optimal policies of finite-state finite-action Markov decision processes (MDPs). A reduction to a continuum of constrained MDPs (CMDPs) is presented such that the optimal policies for these CMDPs constitute a path in a graph defined over the deterministic policies. This path contains, in particular, an optimal policy of the original MDP. We present an algorithm based on this new approach that finds this path, and thus an optimal policy. In the general case, this path might be exponentially long in the number of states and actions. We prove that the length of this path is polynomial if the MDP satisfies a coupling property. Thus we obtain a strongly polynomial algorithm for MDP s that satisfies the coupling property. We prove that discrete time versions of controlled M/M /1 queues induce MDP s that satisfy the coupling property. The only previously known polynomial algorithm for controlled M/M /1 queues in the expected average cost model is based on linear programming (and is not known to be strongly polynomial). Our algorithm works both for the discounted and expected average cost models, and the running time does not depend on the discount factor.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0364-765X
1526-5471
DOI:10.1287/moor.1090.0415