Stochastic Processes with Expected Stopping Time

Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical optimization criterion is the maximal expected total reward wher...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings of the 36th Annual ACM/IEEE Symposium on Logic in Computer Science s. 1 - 13
Hlavní autoři: Chatterjee, Krishnendu, Doyen, Laurent
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 29.06.2021
Témata:
Automata
Complexity theory
Computer science
Heuristic algorithms
Markov processes
Memory management
Probability distribution
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical optimization criterion is the maximal expected total reward where the MDP stops after T steps, which can be computed by a simple dynamic programming algorithm. We consider a natural generalization of the problem where the stopping times can be chosen according to a probability distribution, such that the expected stopping time is T, to optimize the expected total reward. Quite surprisingly we establish inter-reducibility of the expected stopping-time problem for Markov chains with the Positivity problem (which is related to the well-known Skolem problem), for which establishing either decidability or undecidability would be a major breakthrough. Given the hardness of the exact problem, we consider the approximate version of the problem: we show that it can be solved in exponential time for Markov chains and in exponential space for MDPs.
DOI:10.1109/LICS52264.2021.9470595