Scenario-Based Verification of Uncertain MDPs

We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are unknown. The problem is to compute the probability to satisfy a...

Full description

Saved in:
Bibliographic Details
Published in:Tools and algorithms for the construction and analysis of systems : 26th International Conference, TACAS 2020, held as part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2020, Dublin, Ireland, April 25-30 Vol. 12078; p. 287
Main Authors: Cubuktepe, Murat, Jansen, Nils, Junges, Sebastian, Katoen, Joost-Pieter, Topcu, Ufuk
Format: Journal Article
Language:English
Published: Switzerland 01.04.2020
Subjects:
MDP
Uncertainty
Verification
Scenario optimisation
Online Access:Get more information
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are unknown. The problem is to compute the probability to satisfy a temporal logic specification within any MDP that corresponds to a sample from these unknown distributions. In general, this problem is undecidable, and we resort to techniques from so-called scenario optimization. Based on a finite number of samples of the uncertain parameters, each of which induces an MDP, the proposed method estimates the probability of satisfying the specification by solving a finite-dimensional convex optimization problem. The number of samples required to obtain a high confidence on this estimate is independent from the number of states and the number of random parameters. Experiments on a large set of benchmarks show that a few thousand samples suffice to obtain high-quality confidence bounds with a high probability.
AbstractList We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are unknown. The problem is to compute the probability to satisfy a temporal logic specification within any MDP that corresponds to a sample from these unknown distributions. In general, this problem is undecidable, and we resort to techniques from so-called scenario optimization. Based on a finite number of samples of the uncertain parameters, each of which induces an MDP, the proposed method estimates the probability of satisfying the specification by solving a finite-dimensional convex optimization problem. The number of samples required to obtain a high confidence on this estimate is independent from the number of states and the number of random parameters. Experiments on a large set of benchmarks show that a few thousand samples suffice to obtain high-quality confidence bounds with a high probability.We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are unknown. The problem is to compute the probability to satisfy a temporal logic specification within any MDP that corresponds to a sample from these unknown distributions. In general, this problem is undecidable, and we resort to techniques from so-called scenario optimization. Based on a finite number of samples of the uncertain parameters, each of which induces an MDP, the proposed method estimates the probability of satisfying the specification by solving a finite-dimensional convex optimization problem. The number of samples required to obtain a high confidence on this estimate is independent from the number of states and the number of random parameters. Experiments on a large set of benchmarks show that a few thousand samples suffice to obtain high-quality confidence bounds with a high probability.
We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are unknown. The problem is to compute the probability to satisfy a temporal logic specification within any MDP that corresponds to a sample from these unknown distributions. In general, this problem is undecidable, and we resort to techniques from so-called scenario optimization. Based on a finite number of samples of the uncertain parameters, each of which induces an MDP, the proposed method estimates the probability of satisfying the specification by solving a finite-dimensional convex optimization problem. The number of samples required to obtain a high confidence on this estimate is independent from the number of states and the number of random parameters. Experiments on a large set of benchmarks show that a few thousand samples suffice to obtain high-quality confidence bounds with a high probability.
Author Junges, Sebastian
Topcu, Ufuk
Jansen, Nils
Cubuktepe, Murat
Katoen, Joost-Pieter
Author_xml – sequence: 1
  givenname: Murat
  orcidid: 0000-0002-0409-2403
  surname: Cubuktepe
  fullname: Cubuktepe, Murat
  organization: The University of Texas at Austin, Austin, USA
– sequence: 2
  givenname: Nils
  orcidid: 0000-0003-1318-8973
  surname: Jansen
  fullname: Jansen, Nils
  organization: Radboud University Nijmegen, Nijmegen, The Netherlands
– sequence: 3
  givenname: Sebastian
  orcidid: 0000-0003-0978-8466
  surname: Junges
  fullname: Junges, Sebastian
  organization: RWTH Aachen University, Aachen, Germany
– sequence: 4
  givenname: Joost-Pieter
  orcidid: 0000-0002-6143-1926
  surname: Katoen
  fullname: Katoen, Joost-Pieter
  organization: RWTH Aachen University, Aachen, Germany
– sequence: 5
  givenname: Ufuk
  surname: Topcu
  fullname: Topcu, Ufuk
  organization: The University of Texas at Austin, Austin, USA
BackLink https://www.ncbi.nlm.nih.gov/pubmed/32754724$$D View this record in MEDLINE/PubMed
BookMark eNo1j8tKxDAUQLNQfIz-gUiXbqK5eTTtUsfxASMKjm7LneQGAm1ak87Cv1dwXJ3N4cA5ZQdpTMTYBYhrEMLetLbhigsluDbQCm46qI_YsZLWaCv1CePvjhLmOPI7LOSrT8oxRIdzHFM1huojOcozxlS93L-VM3YYsC90vueCbR5Wm-UTX78-Pi9v13xSIGeuDATfaOEdBoXCkArCyjoY37q6Vah93aDXFsiYLbQECqFRrmmDQKm9XLCrv-yUx68dlbkbYnHU95ho3JVOaiVqCwrgV73cq7vtQL6bchwwf3f_h_IHhG1M2w
ContentType Journal Article
DBID NPM
7X8
DOI 10.1007/978-3-030-45190-5_16
DatabaseName PubMed
MEDLINE - Academic
DatabaseTitle PubMed
MEDLINE - Academic
DatabaseTitleList MEDLINE - Academic
PubMed
Database_xml – sequence: 1
  dbid: NPM
  name: PubMed
  url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed
  sourceTypes: Index Database
– sequence: 2
  dbid: 7X8
  name: MEDLINE - Academic
  url: https://search.proquest.com/medline
  sourceTypes: Aggregation Database
DeliveryMethod no_fulltext_linktorsrc
ExternalDocumentID 32754724
Genre Journal Article
GrantInformation_xml – fundername: Shared Services Center NASA
  grantid: 80NSSC19K0209
GroupedDBID NPM
7X8
ID FETCH-LOGICAL-p312t-351fd840dcaf3a05e3f0726f5d9c693a4d68ad471e55b19e13a183c89f0a24d2
IEDL.DBID 7X8
ISICitedReferencesCount 16
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001288732400016&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Fri Jul 11 14:23:09 EDT 2025
Sat May 31 02:13:04 EDT 2025
IsDoiOpenAccess false
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Keywords MDP
Uncertainty
Verification
Scenario optimisation
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-p312t-351fd840dcaf3a05e3f0726f5d9c693a4d68ad471e55b19e13a183c89f0a24d2
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ORCID 0000-0002-6143-1926
0000-0003-0978-8466
0000-0002-0409-2403
0000-0003-1318-8973
OpenAccessLink https://pubmed.ncbi.nlm.nih.gov/PMC7402411
PMID 32754724
PQID 2430671311
PQPubID 23479
ParticipantIDs proquest_miscellaneous_2430671311
pubmed_primary_32754724
PublicationCentury 2000
PublicationDate 2020-Apr
PublicationDateYYYYMMDD 2020-04-01
PublicationDate_xml – month: 04
  year: 2020
  text: 2020-Apr
PublicationDecade 2020
PublicationPlace Switzerland
PublicationPlace_xml – name: Switzerland
PublicationTitle Tools and algorithms for the construction and analysis of systems : 26th International Conference, TACAS 2020, held as part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2020, Dublin, Ireland, April 25-30
PublicationTitleAlternate Tools Algorithms Constr Anal Syst I (2020)
PublicationYear 2020
Score 2.3014596
Snippet We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of...
SourceID proquest
pubmed
SourceType Aggregation Database
Index Database
StartPage 287
Title Scenario-Based Verification of Uncertain MDPs
URI https://www.ncbi.nlm.nih.gov/pubmed/32754724
https://www.proquest.com/docview/2430671311
Volume 12078
WOSCitedRecordID wos001288732400016&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1JS8QwFH6o48GLC27jRgWvwTZbm5O4DR6cYcBR5lbSLOClrXb09_vSdtCLIHjJJQSSvJe87-0AF8qE5A8rSGqsIhwhOCmElkQ6GruMJwX3bRHXx3QyyeZzNe0Nbk0fVrn8E9uP2lYm2MgvKQ_gNhSHuarfSOgaFbyrfQuNVRgwhDKBq9N59iNDrnP-IyOTUEYlJiJP5O9YspUpo63_7mYbNns0GV135N-BFVfuAnkyrkQVuCI3KKJs9IJM5nvTXFT56Bnp3MYBROO7abMHs9H97PaB9F0RSM0Sugih996iWmaN9kzHwjEfp1R6YZWRimluZaYtyhwnRJEolzCNz9Zkyseackv3Ya2sSncIkdUBnwicUYqj2qNZKgpaSOmE1tzpIZwvD58j0wVPgi5d9dHk38cfwkF3g3ndVcfIGU0FTyk_-sPqY9igQX9tI2FOYODxyblTWDefi9fm_aylJo6T6fgLY16p8w
linkProvider ProQuest
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Scenario-Based+Verification+of+Uncertain+MDPs&rft.jtitle=Tools+and+algorithms+for+the+construction+and+analysis+of+systems+%3A+26th+International+Conference%2C+TACAS+2020%2C+held+as+part+of+the+European+Joint+Conferences+on+Theory+and+Practice+of+Software%2C+ETAPS+2020%2C+Dublin%2C+Ireland%2C+April+25-30&rft.au=Cubuktepe%2C+Murat&rft.au=Jansen%2C+Nils&rft.au=Junges%2C+Sebastian&rft.au=Katoen%2C+Joost-Pieter&rft.date=2020-04-01&rft.volume=12078&rft.spage=287&rft_id=info:doi/10.1007%2F978-3-030-45190-5_16&rft_id=info%3Apmid%2F32754724&rft_id=info%3Apmid%2F32754724&rft.externalDocID=32754724