Paradoxes of probabilistic programming: and how to condition on events of measure zero with infinitesimal probabilities

Abstract Probabilistic programming languages allow programmers to write down conditional probability distributions that represent statistical and machine learning models as programs that use observe statements. These programs are run by accumulating likelihood at each observe statement, and using th...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Proceedings of ACM on programming languages Jg. 5; H. POPL; S. 1 - 26
1. Verfasser: Jacobs, Jules
Format: Journal Article
Sprache:Englisch
Veröffentlicht: 01.01.2021
ISSN:2475-1421, 2475-1421
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Abstract Abstract Probabilistic programming languages allow programmers to write down conditional probability distributions that represent statistical and machine learning models as programs that use observe statements. These programs are run by accumulating likelihood at each observe statement, and using the likelihood to steer random choices and weigh results with inference algorithms such as importance sampling or MCMC. We argue that naive likelihood accumulation does not give desirable semantics and leads to paradoxes when an observe statement is used to condition on a measure-zero event, particularly when the observe statement is executed conditionally on random data. We show that the paradoxes disappear if we explicitly model measure-zero events as a limit of positive measure events, and that we can execute these type of probabilistic programs by accumulating infinitesimal probabilities rather than probability densities. Our extension improves probabilistic programming languages as an executable notation for probability distributions by making it more well-behaved and more expressive, by allowing the programmer to be explicit about which limit is intended when conditioning on an event of measure zero.
AbstractList Abstract Probabilistic programming languages allow programmers to write down conditional probability distributions that represent statistical and machine learning models as programs that use observe statements. These programs are run by accumulating likelihood at each observe statement, and using the likelihood to steer random choices and weigh results with inference algorithms such as importance sampling or MCMC. We argue that naive likelihood accumulation does not give desirable semantics and leads to paradoxes when an observe statement is used to condition on a measure-zero event, particularly when the observe statement is executed conditionally on random data. We show that the paradoxes disappear if we explicitly model measure-zero events as a limit of positive measure events, and that we can execute these type of probabilistic programs by accumulating infinitesimal probabilities rather than probability densities. Our extension improves probabilistic programming languages as an executable notation for probability distributions by making it more well-behaved and more expressive, by allowing the programmer to be explicit about which limit is intended when conditioning on an event of measure zero.
Author Jacobs, Jules
Author_xml – sequence: 1
  givenname: Jules
  surname: Jacobs
  fullname: Jacobs, Jules
  organization: Radboud University Nijmegen, Netherlands / Delft University of Technology, Netherlands
BookMark eNplUE1LAzEQDVLBWot_ITdPqztJtm28SdEqFPSg5yWbj3ZkNylJtOqvd6sFRZmBmYH3Hm_eMRn44C0hp1CeA4jqgou-uDwgQyamVQGCweDXfkTGKT2XZQmSixmXQ7J9UFGZ8GYTDY5uYmhUgy2mjHp3raLqOvSrS6q8oeuwpTlQHbzBjMHTvu2r9fmL3FmVXqKlHzYGusW8pugdesw2YafaH_GMNp2QQ6faZMf7OSJPN9eP89tieb-4m18tC82qWS6MBNcIJxhruLHQaGCSVdbp6dS4BsBprVVVMSO5mRihwUqpzAQE9BwwDR-Rs29dHUNK0bp6E3s38b2Gst5FVu8j65HFH6TGrHZv5qiw_Yf_BFnJcd4
CitedBy_id crossref_primary_10_1145_3632170
crossref_primary_10_1145_3649844
Cites_doi 10.1007/978-3-319-23525-7_19
10.1145/3009837.3009852
10.5281/zenodo.4075076
10.1109/LICS.2017.8005137
10.5555/2969033.2969207
10.1145/3371125
10.1111/1467-9574.00056
10.1007/978-3-662-54434-1_32
10.18637/jss.v076.i01
10.1017/S0960129516000098
ContentType Journal Article
DBID AAYXX
CITATION
DOI 10.1145/3434339
DatabaseName CrossRef
DatabaseTitle CrossRef
DatabaseTitleList CrossRef
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISSN 2475-1421
EndPage 26
ExternalDocumentID 10_1145_3434339
GroupedDBID AAKMM
AAYFX
AAYXX
ACM
AEFXT
AEJOY
AIKLT
AKRVB
ALMA_UNASSIGNED_HOLDINGS
CITATION
GUFHI
LHSKQ
M~E
OK1
ROL
ID FETCH-LOGICAL-c258t-d91fb4f422b3de1bc12925efc77dfb11fccca552d93d6d4c1e99ad6141f421db3
ISICitedReferencesCount 4
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000679806100058&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 2475-1421
IngestDate Tue Nov 18 21:52:09 EST 2025
Sat Nov 29 07:52:21 EST 2025
IsDoiOpenAccess false
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue POPL
Language English
LinkModel OpenURL
MergedId FETCHMERGED-LOGICAL-c258t-d91fb4f422b3de1bc12925efc77dfb11fccca552d93d6d4c1e99ad6141f421db3
OpenAccessLink https://dl.acm.org/doi/pdf/10.1145/3434339
PageCount 26
ParticipantIDs crossref_primary_10_1145_3434339
crossref_citationtrail_10_1145_3434339
PublicationCentury 2000
PublicationDate 2021-01-01
PublicationDateYYYYMMDD 2021-01-01
PublicationDate_xml – month: 01
  year: 2021
  text: 2021-01-01
  day: 01
PublicationDecade 2020
PublicationTitle Proceedings of ACM on programming languages
PublicationYear 2021
References Wu Yi (e_1_2_1_16_1) 2018; 80
von Neumann John (e_1_2_1_14_1) 1951; 13
Paige Brooks (e_1_2_1_9_1) 2014
e_1_2_1_7_1
e_1_2_1_5_1
e_1_2_1_6_1
e_1_2_1_3_1
e_1_2_1_12_1
e_1_2_1_4_1
e_1_2_1_1_1
e_1_2_1_10_1
e_1_2_1_2_1
e_1_2_1_11_1
van de Meent Jan-Willem (e_1_2_1_13_1) 1809
Wood Frank (e_1_2_1_15_1)
Jaynes Edwin Thompson (e_1_2_1_8_1)
References_xml – ident: e_1_2_1_12_1
  doi: 10.1007/978-3-319-23525-7_19
– ident: e_1_2_1_10_1
  doi: 10.1145/3009837.3009852
– ident: e_1_2_1_7_1
  doi: 10.5281/zenodo.4075076
– ident: e_1_2_1_6_1
  doi: 10.1109/LICS.2017.8005137
– ident: e_1_2_1_5_1
  doi: 10.5555/2969033.2969207
– start-page: 3410
  year: 2014
  ident: e_1_2_1_9_1
  article-title: Asynchronous Anytime Sequential Monte Carlo. In Advances in Neural Information Processing Systems 27. Curran Associates
  publication-title: Inc.
– ident: e_1_2_1_4_1
  doi: 10.1145/3371125
– ident: e_1_2_1_3_1
  doi: 10.1111/1467-9574.00056
– ident: e_1_2_1_11_1
  doi: 10.1007/978-3-662-54434-1_32
– ident: e_1_2_1_2_1
  doi: 10.18637/jss.v076.i01
– volume-title: Probability theory: The logic of science
  ident: e_1_2_1_8_1
– volume: 13
  start-page: 36
  year: 1951
  ident: e_1_2_1_14_1
  article-title: Various Techniques Used in Connection with Random Digits. In Monte Carlo Method. National Bureau of Standards Applied Mathematics Series, Vol. 12. US Government Printing Ofice, Washington, DC
  publication-title: Chapter
– volume-title: AISTATS 2014 ( JMLR Proceedings). JMLR.org, 1024-1032
  ident: e_1_2_1_15_1
– volume-title: An Introduction to Probabilistic Programming. arXiv:arXiv
  year: 1809
  ident: e_1_2_1_13_1
– volume: 80
  start-page: 5343
  volume-title: Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research
  year: 2018
  ident: e_1_2_1_16_1
– ident: e_1_2_1_1_1
  doi: 10.1017/S0960129516000098
SSID ssj0001934839
Score 2.2050881
Snippet Abstract Probabilistic programming languages allow programmers to write down conditional probability distributions that represent statistical and machine...
SourceID crossref
SourceType Enrichment Source
Index Database
StartPage 1
Title Paradoxes of probabilistic programming: and how to condition on events of measure zero with infinitesimal probabilities
Volume 5
WOSCitedRecordID wos000679806100058&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVHPJ
  databaseName: ROAD: Directory of Open Access Scholarly Resources
  customDbUrl:
  eissn: 2475-1421
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0001934839
  issn: 2475-1421
  databaseCode: M~E
  dateStart: 20170101
  isFulltext: true
  titleUrlDefault: https://road.issn.org
  providerName: ISSN International Centre
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1Lb9QwELaWwoELb9Tykg-IyyqAX0nMrapAHNqSQ5F6Wzl-iJXapNoNy4oDv51x7KTOggQckFbRyvJ4I8-X8exkvhmEXkpnpLTGZbzQNuNCmqzMmcq8805zS7RzPVH4uDg9Lc_PZTWbbQYuzOaiaJpyu5VX_1XVMAbK9tTZf1D3uCgMwHdQOlxB7XD9K8VXaqVMuw3FZH2_mL6Gri_HPCRjXQ4k58bMv_iWcq1PPjfL4Dw2876o0zq8ee8jiPPvdtWGkC3c4dK7qevlpedwDct3Qy5i9HOr8Vzs1zk8OvELJ78_BkqvUxjBNtfryNm2k3AEJUk4ordalBciIzzQnl_b34xFsysSdFWfquPEipLkOA58-l8NPfc1MZjnxYZiSNNS2jtH3Jh4GGjYYhEFb6CbtBDSpwKe_Ehic5Lxsm9CN9554Ft72TdRNnFkEo_k7B66E_9K4MMAgftoZpsH6O7QpgNHq_0QfRsRgVuHJ4jAiUbeYcADBjzgrsUjHjB8Ah68cMQD9njAHg94ggc8wcMj9PnD-7Ojj1nst5FpKsouM5K4mjtOac2MJbUGX5AK63RRGFcT4jQ87kJQI5nJDdfESqkM-HcEZIip2WO017SN3UeY5LB_b5kSDraS5koxKzkD75bSwhGlDtCrYfMWOhaj9z1RLhY7GjpAeJx4Feqv7E558ucpT9Hta6w-Q3vd6qt9jm7pTbdcr170yv8Jb7B-jA
linkProvider ISSN International Centre
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Paradoxes+of+probabilistic+programming%3A+and+how+to+condition+on+events+of+measure+zero+with+infinitesimal+probabilities&rft.jtitle=Proceedings+of+ACM+on+programming+languages&rft.au=Jacobs%2C+Jules&rft.date=2021-01-01&rft.issn=2475-1421&rft.eissn=2475-1421&rft.volume=5&rft.issue=POPL&rft.spage=1&rft.epage=26&rft_id=info:doi/10.1145%2F3434339&rft.externalDBID=n%2Fa&rft.externalDocID=10_1145_3434339
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2475-1421&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2475-1421&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2475-1421&client=summon