Paradoxes of probabilistic programming: and how to condition on events of measure zero with infinitesimal probabilities
Abstract Probabilistic programming languages allow programmers to write down conditional probability distributions that represent statistical and machine learning models as programs that use observe statements. These programs are run by accumulating likelihood at each observe statement, and using th...
Uloženo v:
| Vydáno v: | Proceedings of ACM on programming languages Ročník 5; číslo POPL; s. 1 - 26 |
|---|---|
| Hlavní autor: | |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
01.01.2021
|
| ISSN: | 2475-1421, 2475-1421 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Abstract Probabilistic programming languages allow programmers to write down conditional probability distributions that represent statistical and machine learning models as programs that use observe statements. These programs are run by accumulating likelihood at each observe statement, and using the likelihood to steer random choices and weigh results with inference algorithms such as importance sampling or MCMC. We argue that naive likelihood accumulation does not give desirable semantics and leads to paradoxes when an observe statement is used to condition on a measure-zero event, particularly when the observe statement is executed conditionally on random data. We show that the paradoxes disappear if we explicitly model measure-zero events as a limit of positive measure events, and that we can execute these type of probabilistic programs by accumulating infinitesimal probabilities rather than probability densities. Our extension improves probabilistic programming languages as an executable notation for probability distributions by making it more well-behaved and more expressive, by allowing the programmer to be explicit about which limit is intended when conditioning on an event of measure zero. |
|---|---|
| AbstractList | Abstract Probabilistic programming languages allow programmers to write down conditional probability distributions that represent statistical and machine learning models as programs that use observe statements. These programs are run by accumulating likelihood at each observe statement, and using the likelihood to steer random choices and weigh results with inference algorithms such as importance sampling or MCMC. We argue that naive likelihood accumulation does not give desirable semantics and leads to paradoxes when an observe statement is used to condition on a measure-zero event, particularly when the observe statement is executed conditionally on random data. We show that the paradoxes disappear if we explicitly model measure-zero events as a limit of positive measure events, and that we can execute these type of probabilistic programs by accumulating infinitesimal probabilities rather than probability densities. Our extension improves probabilistic programming languages as an executable notation for probability distributions by making it more well-behaved and more expressive, by allowing the programmer to be explicit about which limit is intended when conditioning on an event of measure zero. |
| Author | Jacobs, Jules |
| Author_xml | – sequence: 1 givenname: Jules surname: Jacobs fullname: Jacobs, Jules organization: Radboud University Nijmegen, Netherlands / Delft University of Technology, Netherlands |
| BookMark | eNplUE1LAzEQDVLBWot_ITdPqztJtm28SdEqFPSg5yWbj3ZkNylJtOqvd6sFRZmBmYH3Hm_eMRn44C0hp1CeA4jqgou-uDwgQyamVQGCweDXfkTGKT2XZQmSixmXQ7J9UFGZ8GYTDY5uYmhUgy2mjHp3raLqOvSrS6q8oeuwpTlQHbzBjMHTvu2r9fmL3FmVXqKlHzYGusW8pugdesw2YafaH_GMNp2QQ6faZMf7OSJPN9eP89tieb-4m18tC82qWS6MBNcIJxhruLHQaGCSVdbp6dS4BsBprVVVMSO5mRihwUqpzAQE9BwwDR-Rs29dHUNK0bp6E3s38b2Gst5FVu8j65HFH6TGrHZv5qiw_Yf_BFnJcd4 |
| CitedBy_id | crossref_primary_10_1145_3632170 crossref_primary_10_1145_3649844 |
| Cites_doi | 10.1007/978-3-319-23525-7_19 10.1145/3009837.3009852 10.5281/zenodo.4075076 10.1109/LICS.2017.8005137 10.5555/2969033.2969207 10.1145/3371125 10.1111/1467-9574.00056 10.1007/978-3-662-54434-1_32 10.18637/jss.v076.i01 10.1017/S0960129516000098 |
| ContentType | Journal Article |
| DBID | AAYXX CITATION |
| DOI | 10.1145/3434339 |
| DatabaseName | CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | CrossRef |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISSN | 2475-1421 |
| EndPage | 26 |
| ExternalDocumentID | 10_1145_3434339 |
| GroupedDBID | AAKMM AAYFX AAYXX ACM AEFXT AEJOY AIKLT AKRVB ALMA_UNASSIGNED_HOLDINGS CITATION GUFHI LHSKQ M~E OK1 ROL |
| ID | FETCH-LOGICAL-c258t-d91fb4f422b3de1bc12925efc77dfb11fccca552d93d6d4c1e99ad6141f421db3 |
| ISICitedReferencesCount | 4 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000679806100058&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2475-1421 |
| IngestDate | Tue Nov 18 21:52:09 EST 2025 Sat Nov 29 07:52:21 EST 2025 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | POPL |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c258t-d91fb4f422b3de1bc12925efc77dfb11fccca552d93d6d4c1e99ad6141f421db3 |
| OpenAccessLink | https://dl.acm.org/doi/pdf/10.1145/3434339 |
| PageCount | 26 |
| ParticipantIDs | crossref_primary_10_1145_3434339 crossref_citationtrail_10_1145_3434339 |
| PublicationCentury | 2000 |
| PublicationDate | 2021-01-01 |
| PublicationDateYYYYMMDD | 2021-01-01 |
| PublicationDate_xml | – month: 01 year: 2021 text: 2021-01-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings of ACM on programming languages |
| PublicationYear | 2021 |
| References | Wu Yi (e_1_2_1_16_1) 2018; 80 von Neumann John (e_1_2_1_14_1) 1951; 13 Paige Brooks (e_1_2_1_9_1) 2014 e_1_2_1_7_1 e_1_2_1_5_1 e_1_2_1_6_1 e_1_2_1_3_1 e_1_2_1_12_1 e_1_2_1_4_1 e_1_2_1_1_1 e_1_2_1_10_1 e_1_2_1_2_1 e_1_2_1_11_1 van de Meent Jan-Willem (e_1_2_1_13_1) 1809 Wood Frank (e_1_2_1_15_1) Jaynes Edwin Thompson (e_1_2_1_8_1) |
| References_xml | – ident: e_1_2_1_12_1 doi: 10.1007/978-3-319-23525-7_19 – ident: e_1_2_1_10_1 doi: 10.1145/3009837.3009852 – ident: e_1_2_1_7_1 doi: 10.5281/zenodo.4075076 – ident: e_1_2_1_6_1 doi: 10.1109/LICS.2017.8005137 – ident: e_1_2_1_5_1 doi: 10.5555/2969033.2969207 – start-page: 3410 year: 2014 ident: e_1_2_1_9_1 article-title: Asynchronous Anytime Sequential Monte Carlo. In Advances in Neural Information Processing Systems 27. Curran Associates publication-title: Inc. – ident: e_1_2_1_4_1 doi: 10.1145/3371125 – ident: e_1_2_1_3_1 doi: 10.1111/1467-9574.00056 – ident: e_1_2_1_11_1 doi: 10.1007/978-3-662-54434-1_32 – ident: e_1_2_1_2_1 doi: 10.18637/jss.v076.i01 – volume-title: Probability theory: The logic of science ident: e_1_2_1_8_1 – volume: 13 start-page: 36 year: 1951 ident: e_1_2_1_14_1 article-title: Various Techniques Used in Connection with Random Digits. In Monte Carlo Method. National Bureau of Standards Applied Mathematics Series, Vol. 12. US Government Printing Ofice, Washington, DC publication-title: Chapter – volume-title: AISTATS 2014 ( JMLR Proceedings). JMLR.org, 1024-1032 ident: e_1_2_1_15_1 – volume-title: An Introduction to Probabilistic Programming. arXiv:arXiv year: 1809 ident: e_1_2_1_13_1 – volume: 80 start-page: 5343 volume-title: Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research year: 2018 ident: e_1_2_1_16_1 – ident: e_1_2_1_1_1 doi: 10.1017/S0960129516000098 |
| SSID | ssj0001934839 |
| Score | 2.2050881 |
| Snippet | Abstract Probabilistic programming languages allow programmers to write down conditional probability distributions that represent statistical and machine... |
| SourceID | crossref |
| SourceType | Enrichment Source Index Database |
| StartPage | 1 |
| Title | Paradoxes of probabilistic programming: and how to condition on events of measure zero with infinitesimal probabilities |
| Volume | 5 |
| WOSCitedRecordID | wos000679806100058&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2475-1421 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001934839 issn: 2475-1421 databaseCode: M~E dateStart: 20170101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1La9wwEBbbpIde0qYPkjYtOoReFreRLa2s3kJIyCEPH1LIbbFedCGxw66zXXrob-_Ikh15W2h7CBhjhEY2O59H49n5ZhDal7ANGZYfJPBxYROqtUpya2UCvjKXQroSWnnbbIJfXOTX16IYjZYdF2Z5w6sqX63E3aOqGsZA2Y46-x_q7heFAbgGpcMZ1A7nf1J8Uc5LXa98MVnXL6atoevKMXfJWLcdybnS42-upVztks_1zDuP1bgt6rTw_7y3EcTxDzOvfcgWnnDm3NTF7NZxuLrlmy4XMfi5Rb8vtuscHp27haP794HShxRGsM1yETjbZhCOSEkUjmitVko5Swj1tOdP5g9jweyyCF3FZXEWWVESbceeT_-7oaeuJkbmeLG-GNKwlPbaFtcnHnoaNpsGwSdoM-VMuFTA859RbE5kNG-b0PVP7vnWTvZzkI0cmcgjuXqBtsKnBD70ENhGI1O9RM-7Nh04WO1X6HuPCFxbPEAEjjTyBQMeMOABNzXu8YDh8HhwwgEP2OEBOzzgAR7wAA-v0deT46uj0yT020hUyvIm0YJYSS1NU5lpQ6QCXzBlxirOtZXwLit43RlLtcj0RFNFjBClBv-OgAzRMnuDNqq6MjsIg9eoJibTxEhFpSW5kZzJgzIrywm3jO6ij92PN1WhGL3riXIzXdPQLsL9xDtff2V9ytu_T3mHnj1gdQ9tNPN78x49Vctmtph_aJX_C1cvfy8 |
| linkProvider | ISSN International Centre |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Paradoxes+of+probabilistic+programming%3A+and+how+to+condition+on+events+of+measure+zero+with+infinitesimal+probabilities&rft.jtitle=Proceedings+of+ACM+on+programming+languages&rft.au=Jacobs%2C+Jules&rft.date=2021-01-01&rft.issn=2475-1421&rft.eissn=2475-1421&rft.volume=5&rft.issue=POPL&rft.spage=1&rft.epage=26&rft_id=info:doi/10.1145%2F3434339&rft.externalDBID=n%2Fa&rft.externalDocID=10_1145_3434339 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2475-1421&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2475-1421&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2475-1421&client=summon |