Decision-Dependent Distributionally Robust Markov Decision Process Method in Dynamic Epidemic Control

In this paper, we present a Distributionally Robust Markov Decision Process (DRMDP) approach for addressing the dynamic epidemic control problem. The Susceptible-Exposed-Infectious-Recovered (SEIR) model is widely used to represent the stochastic spread of infectious diseases, such as COVID-19. Whil...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:arXiv.org
Hlavní autori: Song, Jun, Yang, William, Zhao, Chaoyue
Médium: Paper
Jazyk:English
Vydavateľské údaje: Ithaca Cornell University Library, arXiv.org 24.06.2023
Predmet:
Algorithms
Disease control
Dynamic programming
Epidemics
Infectious diseases
Markov analysis
Markov processes
Real-time programming
Robustness (mathematics)
Transition probabilities
ISSN:2331-8422
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Abstract In this paper, we present a Distributionally Robust Markov Decision Process (DRMDP) approach for addressing the dynamic epidemic control problem. The Susceptible-Exposed-Infectious-Recovered (SEIR) model is widely used to represent the stochastic spread of infectious diseases, such as COVID-19. While Markov Decision Processes (MDP) offers a mathematical framework for identifying optimal actions, such as vaccination and transmission-reducing intervention, to combat disease spreading according to the SEIR model. However, uncertainties in these scenarios demand a more robust approach that is less reliant on error-prone assumptions. The primary objective of our study is to introduce a new DRMDP framework that allows for an ambiguous distribution of transition dynamics. Specifically, we consider the worst-case distribution of these transition probabilities within a decision-dependent ambiguity set. To overcome the computational complexities associated with policy determination, we propose an efficient Real-Time Dynamic Programming (RTDP) algorithm that is capable of computing optimal policies based on the reformulated DRMDP model in an accurate, timely, and scalable manner. Comparative analysis against the classic MDP model demonstrates that the DRMDP achieves a lower proportion of infections and susceptibilities at a reduced cost.
AbstractList In this paper, we present a Distributionally Robust Markov Decision Process (DRMDP) approach for addressing the dynamic epidemic control problem. The Susceptible-Exposed-Infectious-Recovered (SEIR) model is widely used to represent the stochastic spread of infectious diseases, such as COVID-19. While Markov Decision Processes (MDP) offers a mathematical framework for identifying optimal actions, such as vaccination and transmission-reducing intervention, to combat disease spreading according to the SEIR model. However, uncertainties in these scenarios demand a more robust approach that is less reliant on error-prone assumptions. The primary objective of our study is to introduce a new DRMDP framework that allows for an ambiguous distribution of transition dynamics. Specifically, we consider the worst-case distribution of these transition probabilities within a decision-dependent ambiguity set. To overcome the computational complexities associated with policy determination, we propose an efficient Real-Time Dynamic Programming (RTDP) algorithm that is capable of computing optimal policies based on the reformulated DRMDP model in an accurate, timely, and scalable manner. Comparative analysis against the classic MDP model demonstrates that the DRMDP achieves a lower proportion of infections and susceptibilities at a reduced cost.
Author Song, Jun
Yang, William
Zhao, Chaoyue
Author_xml – sequence: 1
  givenname: Jun
  surname: Song
  fullname: Song, Jun
– sequence: 2
  givenname: William
  surname: Yang
  fullname: Yang, William
– sequence: 3
  givenname: Chaoyue
  surname: Zhao
  fullname: Zhao, Chaoyue
BookMark eNo1j99LwzAcxIMoOOf-AN8CPnfmZ5s-SjunsKHI3kfafoOZNalJOtx_b0V9uuPgc9xdoXPnHSB0Q8lSKCnJnQ5f9rhknORLKoikZ2jGOKeZEoxdokWMB0IIywsmJZ8hqKG10XqX1TCA68AlXNuYgm3GNMW670_41TdjTHirw7s_4n8CvwTfQox4C-nNd9g6XJ-c_rAtXg22gx9TeZeC76_RhdF9hMWfztHuYbWrHrPN8_qput9kWjKadaSYBjMuaatbqQQVqtAl1bKklBTSEGKK0nAOwijZiJwpZXJQJedMGZjAObr9rR2C_xwhpv3Bj2H6EPdMsbJUE0L5NwHbWO8
ContentType Paper
Copyright 2023. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Copyright_xml – notice: 2023. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
DBID 8FE
8FG
ABJCF
ABUWG
AFKRA
AZQEC
BENPR
BGLVJ
CCPQU
COVID
DWQXO
HCIFZ
L6V
M7S
PHGZM
PHGZT
PIMPY
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
PTHSS
DOI 10.48550/arxiv.2306.14051
DatabaseName ProQuest SciTech Collection
ProQuest Technology Collection
ProQuest Materials Science & Engineering
ProQuest Central (Alumni)
ProQuest Central UK/Ireland
ProQuest Central Essentials
ProQuest Central
Technology collection
ProQuest One
Coronavirus Research Database
ProQuest Central
SciTech Premium Collection
ProQuest Engineering Collection
Engineering Database
ProQuest Central Premium
ProQuest One Academic
Publicly Available Content Database
ProQuest One Academic Middle East (New)
ProQuest One Academic Eastern Edition (DO NOT USE)
ProQuest One Applied & Life Sciences
ProQuest One Academic (retired)
ProQuest One Academic UKI Edition
ProQuest Central China
Engineering collection
DatabaseTitle Publicly Available Content Database
Engineering Database
Technology Collection
ProQuest One Academic Middle East (New)
ProQuest Central Essentials
ProQuest One Academic Eastern Edition
Coronavirus Research Database
ProQuest Central (Alumni Edition)
SciTech Premium Collection
ProQuest One Community College
ProQuest Technology Collection
ProQuest SciTech Collection
ProQuest Central China
ProQuest Central
ProQuest One Applied & Life Sciences
ProQuest Engineering Collection
ProQuest One Academic UKI Edition
ProQuest Central Korea
Materials Science & Engineering Collection
ProQuest Central (New)
ProQuest One Academic
ProQuest One Academic (New)
Engineering Collection
DatabaseTitleList Publicly Available Content Database
Database_xml – sequence: 1
  dbid: PIMPY
  name: Publicly Available Content Database
  url: http://search.proquest.com/publiccontent
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Physics
EISSN 2331-8422
Genre Working Paper/Pre-Print
GroupedDBID 8FE
8FG
ABJCF
ABUWG
AFKRA
ALMA_UNASSIGNED_HOLDINGS
AZQEC
BENPR
BGLVJ
CCPQU
COVID
DWQXO
FRJ
HCIFZ
L6V
M7S
M~E
PHGZM
PHGZT
PIMPY
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
PTHSS
ID FETCH-LOGICAL-a521-d074052351cac5841487a91a5911075f00f79f33e4f85b46288f6e893328fe523
IEDL.DBID M7S
IngestDate Mon Jun 30 09:11:08 EDT 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a521-d074052351cac5841487a91a5911075f00f79f33e4f85b46288f6e893328fe523
Notes SourceType-Working Papers-1
ObjectType-Working Paper/Pre-Print-1
content type line 50
OpenAccessLink https://www.proquest.com/docview/2829986281?pq-origsite=%requestingapplication%
PQID 2829986281
PQPubID 2050157
ParticipantIDs proquest_journals_2829986281
PublicationCentury 2000
PublicationDate 20230624
PublicationDateYYYYMMDD 2023-06-24
PublicationDate_xml – month: 06
  year: 2023
  text: 20230624
  day: 24
PublicationDecade 2020
PublicationPlace Ithaca
PublicationPlace_xml – name: Ithaca
PublicationTitle arXiv.org
PublicationYear 2023
Publisher Cornell University Library, arXiv.org
Publisher_xml – name: Cornell University Library, arXiv.org
SSID ssj0002672553
Score 1.835863
SecondaryResourceType preprint
Snippet In this paper, we present a Distributionally Robust Markov Decision Process (DRMDP) approach for addressing the dynamic epidemic control problem. The...
SourceID proquest
SourceType Aggregation Database
SubjectTerms Algorithms
Disease control
Dynamic programming
Epidemics
Infectious diseases
Markov analysis
Markov processes
Real-time programming
Robustness (mathematics)
Transition probabilities
Title Decision-Dependent Distributionally Robust Markov Decision Process Method in Dynamic Epidemic Control
URI https://www.proquest.com/docview/2829986281
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LS8NAEF60VfDkGx-17MHr0mxem5wEmxYFW0ItUk9lsw8olKRN0qL_3t3NVg-CF48hLIQNM_PNzDfzAXAvIx5wLxQoJFwiFSGUSblMIsKIsi8RY9-wLd5eyHgczWZxagtulaVV7nyicdS8YLpG3tMdv1jB7wg_rNZIq0bp7qqV0NgHbb0lARvq3ut3jcUNiULMXtPMNKu7erT8WGw1-zlUPsIJ8C8XbOLK8Pi_X3QC2ildifIU7In8DBwaPierzoFIrHYOSqzKbQ0TvSPXylvR5fITTopsU9VQj-sUW7g7Ae3sABwZcWm4yGHSyNbDQSMny2C_IbhfgOlwMO0_IauogKgK04grvKDLwAFmlCnkoVIhQmNMg1hngYF0HEli6XnCl1GQ6anVSIZCIRrPjaRQBy9BKy9ycQWgr_IQx-OZ1LuFMSEZZS6PhcAB4a4fhtegs7u0ubWKav5zYzd_v74FR1rWXVOyXL8DWnW5EXfggG3rRVV2QftxME4nXfOz1VP6PErfvwDoV7Rx
linkProvider ProQuest
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V05T8MwFLaqFgQTtzgKeIAxauMcjgfE0LRq1UMVVKhb5Di2FKkkJUkL_VH8R-wcMCCxdWC2MjjP7_7e-wC4E05gBYbNNRsHQpMeQqoUYkLDDEv94kQ3c7TFywhPJs58TqY18FnNwihYZWUTc0MdxEzVyFuq40dk-O3oj8s3TbFGqe5qRaFRPIsh37zLlC19GLhSvvcI9bqzTl8rWQU0Kl2VFkifqUqhls4ok95XpgOYEp1aRGVClmi3BSbCMLgpHMtXk5uOsLkipUeO4JbacyAtfkNGEYjkSMHn75IOsrEM0I2id5pvCmvR5CNcK7C1LU1S29J_WfzcjfUO_tkPOASNKV3y5AjUeHQMdnO0KktPAHdLZiDNLTl8M-iqDcAleRddLDbwKfZXaQbVMFK8htUXsJyMgOOcOhuGEXQ3EX0NGewWZLkMdgr4_imYbeNiZ6AexRE_B9CUWVbbCHyhNifrGPuUoYBwrls4QKZtX4BmJSOv1PnU-xHQ5d_Ht2CvPxuPvNFgMrwC-4rAXoHPkNkE9SxZ8Wuww9ZZmCY3-fuCwNuyOL8Av64IIg
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Decision-Dependent+Distributionally+Robust+Markov+Decision+Process+Method+in+Dynamic+Epidemic+Control&rft.jtitle=arXiv.org&rft.au=Song%2C+Jun&rft.au=Yang%2C+William&rft.au=Zhao%2C+Chaoyue&rft.date=2023-06-24&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422&rft_id=info:doi/10.48550%2Farxiv.2306.14051