An Actor-Critic Algorithm with Function Approximation for Risk Sensitive Cost Markov Decision Processes

In this paper, we consider the risk-sensitive cost criterion with exponentiated costs for Markov decision processes and develop a model-free policy gradient algorithm in this setting. Unlike additive cost criteria such as average or discounted cost, the risk-sensitive cost criterion is less studied...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transactions on automatic control s. 1 - 8
Hlavní autoři: Guin, Soumyajit, Borkar, Vivek S., Bhatnagar, Shalabh
Médium: Journal Article
Jazyk:angličtina
Vydáno: IEEE 2025
Témata:
Approximation algorithms
Convergence
Costs
Function approximation
Machine learning algorithms
Markov decision processes
Training
Vectors
Zinc
ISSN:0018-9286, 1558-2523
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:In this paper, we consider the risk-sensitive cost criterion with exponentiated costs for Markov decision processes and develop a model-free policy gradient algorithm in this setting. Unlike additive cost criteria such as average or discounted cost, the risk-sensitive cost criterion is less studied due to the complexity resulting from the multiplicative structure of the resulting Bellman equation. We develop an actor-critic algorithm with function approximation in this setting and provide its asymptotic convergence analysis. We also show the results of numerical experiments that demonstrate the superiority in performance of our algorithm over other recent algorithms in the literature.
ISSN:0018-9286
1558-2523
DOI:10.1109/TAC.2025.3593328