Joint chance-constrained Markov decision processes

We consider a finite state-action uncertain constrained Markov decision process under discounted and average cost criteria. The running costs are defined by random variables and the transition probabilities are known. The uncertainties present in the objective function and the constraints are modell...

Full description

Saved in:
Bibliographic Details
Published in:Annals of operations research Vol. 322; no. 2; pp. 1013 - 1035
Main Authors: Varagapriya, V, Singh, Vikas Vikram, Lisser, Abdel
Format: Journal Article
Language:English
Published: New York Springer US 01.03.2023
Springer Nature B.V
Springer Verlag
Subjects:
Admission control
Approximation
Business and Management
Combinatorics
Costs
Decision making
Linear programming
Markov analysis
Markov processes
Mathematics
Operations research
Operations Research/Decision Theory
Optimization
Original Research
Probability
Queuing theory
Random variables
Theory of Computation
Transition probabilities
Upper bounds
Joint chance constraint
Copula
Constrained Markov decision process
Elliptical distribution
Queueing problem
Second order cone programming problem
ISSN:0254-5330, 1572-9338
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We consider a finite state-action uncertain constrained Markov decision process under discounted and average cost criteria. The running costs are defined by random variables and the transition probabilities are known. The uncertainties present in the objective function and the constraints are modelled using chance constraints. We assume that the random cost vectors follow multivariate elliptically symmetric distributions and dependence among the random constraints is driven by a Gumbel–Hougaard copula. We propose two second order cone programming problems whose optimal values give lower and upper bounds of the optimal value of the uncertain constrained Markov decision process. As an application, we study a stochastic version of a service and admission control problem in a queueing system. The proposed approximation methods are illustrated on randomly generated instances of queueing control problem as well as on well known class of Markov decision problems known as Garnets.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0254-5330
1572-9338
DOI:10.1007/s10479-022-05025-3