Scenario-Based Verification of Uncertain MDPs
We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are unknown. The problem is to compute the probability to satisfy a...
Uloženo v:
| Vydáno v: | Tools and algorithms for the construction and analysis of systems : 26th International Conference, TACAS 2020, held as part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2020, Dublin, Ireland, April 25-30 Ročník 12078; s. 287 |
|---|---|
| Hlavní autoři: | , , , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
Switzerland
01.04.2020
|
| Témata: | |
| On-line přístup: | Zjistit podrobnosti o přístupu |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are unknown. The problem is to compute the probability to satisfy a temporal logic specification within any MDP that corresponds to a sample from these unknown distributions. In general, this problem is undecidable, and we resort to techniques from so-called scenario optimization. Based on a finite number of samples of the uncertain parameters, each of which induces an MDP, the proposed method estimates the probability of satisfying the specification by solving a finite-dimensional convex optimization problem. The number of samples required to obtain a high confidence on this estimate is independent from the number of states and the number of random parameters. Experiments on a large set of benchmarks show that a few thousand samples suffice to obtain high-quality confidence bounds with a high probability. |
|---|---|
| AbstractList | We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are unknown. The problem is to compute the probability to satisfy a temporal logic specification within any MDP that corresponds to a sample from these unknown distributions. In general, this problem is undecidable, and we resort to techniques from so-called scenario optimization. Based on a finite number of samples of the uncertain parameters, each of which induces an MDP, the proposed method estimates the probability of satisfying the specification by solving a finite-dimensional convex optimization problem. The number of samples required to obtain a high confidence on this estimate is independent from the number of states and the number of random parameters. Experiments on a large set of benchmarks show that a few thousand samples suffice to obtain high-quality confidence bounds with a high probability.We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are unknown. The problem is to compute the probability to satisfy a temporal logic specification within any MDP that corresponds to a sample from these unknown distributions. In general, this problem is undecidable, and we resort to techniques from so-called scenario optimization. Based on a finite number of samples of the uncertain parameters, each of which induces an MDP, the proposed method estimates the probability of satisfying the specification by solving a finite-dimensional convex optimization problem. The number of samples required to obtain a high confidence on this estimate is independent from the number of states and the number of random parameters. Experiments on a large set of benchmarks show that a few thousand samples suffice to obtain high-quality confidence bounds with a high probability. We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are unknown. The problem is to compute the probability to satisfy a temporal logic specification within any MDP that corresponds to a sample from these unknown distributions. In general, this problem is undecidable, and we resort to techniques from so-called scenario optimization. Based on a finite number of samples of the uncertain parameters, each of which induces an MDP, the proposed method estimates the probability of satisfying the specification by solving a finite-dimensional convex optimization problem. The number of samples required to obtain a high confidence on this estimate is independent from the number of states and the number of random parameters. Experiments on a large set of benchmarks show that a few thousand samples suffice to obtain high-quality confidence bounds with a high probability. |
| Author | Junges, Sebastian Topcu, Ufuk Jansen, Nils Cubuktepe, Murat Katoen, Joost-Pieter |
| Author_xml | – sequence: 1 givenname: Murat orcidid: 0000-0002-0409-2403 surname: Cubuktepe fullname: Cubuktepe, Murat organization: The University of Texas at Austin, Austin, USA – sequence: 2 givenname: Nils orcidid: 0000-0003-1318-8973 surname: Jansen fullname: Jansen, Nils organization: Radboud University Nijmegen, Nijmegen, The Netherlands – sequence: 3 givenname: Sebastian orcidid: 0000-0003-0978-8466 surname: Junges fullname: Junges, Sebastian organization: RWTH Aachen University, Aachen, Germany – sequence: 4 givenname: Joost-Pieter orcidid: 0000-0002-6143-1926 surname: Katoen fullname: Katoen, Joost-Pieter organization: RWTH Aachen University, Aachen, Germany – sequence: 5 givenname: Ufuk surname: Topcu fullname: Topcu, Ufuk organization: The University of Texas at Austin, Austin, USA |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/32754724$$D View this record in MEDLINE/PubMed |
| BookMark | eNo1j8tKxDAUQLNQfIz-gUiXbqK5eTTtUsfxASMKjm7LneQGAm1ak87Cv1dwXJ3N4cA5ZQdpTMTYBYhrEMLetLbhigsluDbQCm46qI_YsZLWaCv1CePvjhLmOPI7LOSrT8oxRIdzHFM1huojOcozxlS93L-VM3YYsC90vueCbR5Wm-UTX78-Pi9v13xSIGeuDATfaOEdBoXCkArCyjoY37q6Vah93aDXFsiYLbQECqFRrmmDQKm9XLCrv-yUx68dlbkbYnHU95ho3JVOaiVqCwrgV73cq7vtQL6bchwwf3f_h_IHhG1M2w |
| ContentType | Journal Article |
| DBID | NPM 7X8 |
| DOI | 10.1007/978-3-030-45190-5_16 |
| DatabaseName | PubMed MEDLINE - Academic |
| DatabaseTitle | PubMed MEDLINE - Academic |
| DatabaseTitleList | MEDLINE - Academic PubMed |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database |
| DeliveryMethod | no_fulltext_linktorsrc |
| ExternalDocumentID | 32754724 |
| Genre | Journal Article |
| GrantInformation_xml | – fundername: Shared Services Center NASA grantid: 80NSSC19K0209 |
| GroupedDBID | NPM 7X8 |
| ID | FETCH-LOGICAL-p312t-351fd840dcaf3a05e3f0726f5d9c693a4d68ad471e55b19e13a183c89f0a24d2 |
| IEDL.DBID | 7X8 |
| ISICitedReferencesCount | 16 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001288732400016&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Fri Jul 11 14:23:09 EDT 2025 Sat May 31 02:13:04 EDT 2025 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | MDP Uncertainty Verification Scenario optimisation |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-p312t-351fd840dcaf3a05e3f0726f5d9c693a4d68ad471e55b19e13a183c89f0a24d2 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
| ORCID | 0000-0002-6143-1926 0000-0003-0978-8466 0000-0002-0409-2403 0000-0003-1318-8973 |
| OpenAccessLink | https://pubmed.ncbi.nlm.nih.gov/PMC7402411 |
| PMID | 32754724 |
| PQID | 2430671311 |
| PQPubID | 23479 |
| ParticipantIDs | proquest_miscellaneous_2430671311 pubmed_primary_32754724 |
| PublicationCentury | 2000 |
| PublicationDate | 2020-Apr |
| PublicationDateYYYYMMDD | 2020-04-01 |
| PublicationDate_xml | – month: 04 year: 2020 text: 2020-Apr |
| PublicationDecade | 2020 |
| PublicationPlace | Switzerland |
| PublicationPlace_xml | – name: Switzerland |
| PublicationTitle | Tools and algorithms for the construction and analysis of systems : 26th International Conference, TACAS 2020, held as part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2020, Dublin, Ireland, April 25-30 |
| PublicationTitleAlternate | Tools Algorithms Constr Anal Syst I (2020) |
| PublicationYear | 2020 |
| Score | 2.3014596 |
| Snippet | We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of... |
| SourceID | proquest pubmed |
| SourceType | Aggregation Database Index Database |
| StartPage | 287 |
| Title | Scenario-Based Verification of Uncertain MDPs |
| URI | https://www.ncbi.nlm.nih.gov/pubmed/32754724 https://www.proquest.com/docview/2430671311 |
| Volume | 12078 |
| WOSCitedRecordID | wos001288732400016&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LS8QwEA7qevDiA1_riwpeg23SJM1JfC0e3GXBVfZW0kwCXtpqV3-_k7aLXgTBS26BJJNkvpn5ZoaQiwK1mmU-pP3ymIYaUtRY0BSkkRZEBlC01fUf1WSSzed62jvcmp5WufwT248aKht85JcsDeA2FIe5qt9o6BoVoqt9C41VMuAIZQKlS82zHxlyXfA_DssQIWla5In8HUu2OmW09d_VbJPNHk1G1534d8iKK3cJfbKuRBO4ojeooiB6wUvme9dcVPnoGeXc8gCi8d202SOz0f3s9oH2XRFozRO2CNR7D2iWgTWem1g47mPFpBegrdTcpCAzA6hznBBFol3CDT5bm2kfG5YC2ydrZVW6QxIZ6axMtQXOHeIopZPCGGsBIYQyDviQnC83n-OlC5EEU7rqo8m_tz8kB90J5nVXHSPnTIlUsfToD7OPyQYL9mvLhDkhA49Pzp2Sdfu5eG3ez1pp4jiZjr8AB9mrKg |
| linkProvider | ProQuest |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Scenario-Based+Verification+of+Uncertain+MDPs&rft.jtitle=Tools+and+algorithms+for+the+construction+and+analysis+of+systems+%3A+26th+International+Conference%2C+TACAS+2020%2C+held+as+part+of+the+European+Joint+Conferences+on+Theory+and+Practice+of+Software%2C+ETAPS+2020%2C+Dublin%2C+Ireland%2C+April+25-30&rft.au=Cubuktepe%2C+Murat&rft.au=Jansen%2C+Nils&rft.au=Junges%2C+Sebastian&rft.au=Katoen%2C+Joost-Pieter&rft.date=2020-04-01&rft.volume=12078&rft.spage=287&rft_id=info:doi/10.1007%2F978-3-030-45190-5_16&rft_id=info%3Apmid%2F32754724&rft_id=info%3Apmid%2F32754724&rft.externalDocID=32754724 |