Quantile Markov Decision Processes

The goal of a traditional Markov decision process (MDP) is to maximize expected cumulative reward over a defined horizon (possibly infinite). In many applications, however, a decision maker may be interested in optimizing a specific quantile of the cumulative reward instead of its expectation. In th...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	Operations research Ročník 70; číslo 3; s. 1428
Hlavní autoři:	Li, Xiaocheng, Zhong, Huaiyang, Brandeau, Margaret L
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	United States 01.05.2022
Témata:	Markov Decision Process Dynamic Programming Risk Measure Quantile Medical Decision Making
ISSN:	0030-364X
On-line přístup:	Zjistit podrobnosti o přístupu
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Popis
Shrnutí:	The goal of a traditional Markov decision process (MDP) is to maximize expected cumulative reward over a defined horizon (possibly infinite). In many applications, however, a decision maker may be interested in optimizing a specific quantile of the cumulative reward instead of its expectation. In this paper we consider the problem of optimizing the quantiles of the cumulative rewards of a Markov decision process (MDP), which we refer to as a quantile Markov decision process (QMDP). We provide analytical results characterizing the optimal QMDP value function and present a dynamic programming-based algorithm to solve for the optimal policy. The algorithm also extends to the MDP problem with a conditional value-at-risk (CVaR) objective. We illustrate the practical relevance of our model by evaluating it on an HIV treatment initiation problem, where patients aim to balance the potential benefits and risks of the treatment.
Bibliografie:	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23
ISSN:	0030-364X
DOI:	10.1287/opre.2021.2123