A martingale approach and time-consistent sampling-based algorithms for risk management in stochastic optimal control

In this paper, we consider a class of stochastic optimal control problems with risk constraints that are expressed as bounded probabilities of failure for particular initial states. We present here a martingale approach that diffuses a risk constraint into a martingale to construct time-consistent c...

Full description

Saved in:
Bibliographic Details
Published in:53rd IEEE Conference on Decision and Control pp. 1858 - 1865
Main Authors: Vu Anh Huynh, Kogan, Leonid, Frazzoli, Emilio
Format: Conference Proceeding
Language:English
Published: IEEE 01.12.2014
Subjects:
Approximation algorithms
Approximation methods
Manganese
Markov processes
Optimal control
Process control
Tin
ISBN:9781479977468, 1479977462
ISSN:0191-2216
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this paper, we consider a class of stochastic optimal control problems with risk constraints that are expressed as bounded probabilities of failure for particular initial states. We present here a martingale approach that diffuses a risk constraint into a martingale to construct time-consistent control policies. The martingale stands for the level of risk tolerance that is contingent on available information over time. By augmenting the system dynamics with the controlled martingale, the original risk-constrained problem is transformed into a stochastic target problem. We extend the incremental Markov Decision Process (iMDP) algorithm to approximate arbitrarily well an optimal feedback policy of the original problem by sampling in the augmented state space and computing proper boundary conditions for the reformulated problem. We show that the algorithm is both probabilistically sound and asymptotically optimal. The performance of the proposed algorithm is demonstrated on motion planning and control problems subject to bounded probability of collision in uncertain cluttered environments.
ISBN:9781479977468
1479977462
ISSN:0191-2216
DOI:10.1109/CDC.2014.7039669