On the marginal likelihood and cross-validation
Summary In Bayesian statistics, the marginal likelihood, also known as the evidence, is used to evaluate model fit as it quantifies the joint probability of the data under the prior. In contrast, non-Bayesian models are typically compared using cross-validation on held-out data, either through $k$-f...
Uloženo v:
| Vydáno v: | Biometrika Ročník 107; číslo 2; s. 489 - 496 |
|---|---|
| Hlavní autoři: | , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
Oxford University Press
01.06.2020
|
| Témata: | |
| ISSN: | 0006-3444, 1464-3510 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Summary
In Bayesian statistics, the marginal likelihood, also known as the evidence, is used to evaluate model fit as it quantifies the joint probability of the data under the prior. In contrast, non-Bayesian models are typically compared using cross-validation on held-out data, either through $k$-fold partitioning or leave-$p$-out subsampling. We show that the marginal likelihood is formally equivalent to exhaustive leave-$p$-out crossvalidation averaged over all values of $p$ and all held-out test sets when using the log posterior predictive probability as the scoring rule. Moreover, the log posterior predictive score is the only coherent scoring rule under data exchangeability. This offers new insight into the marginal likelihood and cross-validation, and highlights the potential sensitivity of the marginal likelihood to the choice of the prior. We suggest an alternative approach using cumulative cross-validation following a preparatory training phase. Our work has connections to prequential analysis and intrinsic Bayes factors, but is motivated in a different way. |
|---|---|
| AbstractList | Summary
In Bayesian statistics, the marginal likelihood, also known as the evidence, is used to evaluate model fit as it quantifies the joint probability of the data under the prior. In contrast, non-Bayesian models are typically compared using cross-validation on held-out data, either through $k$-fold partitioning or leave-$p$-out subsampling. We show that the marginal likelihood is formally equivalent to exhaustive leave-$p$-out crossvalidation averaged over all values of $p$ and all held-out test sets when using the log posterior predictive probability as the scoring rule. Moreover, the log posterior predictive score is the only coherent scoring rule under data exchangeability. This offers new insight into the marginal likelihood and cross-validation, and highlights the potential sensitivity of the marginal likelihood to the choice of the prior. We suggest an alternative approach using cumulative cross-validation following a preparatory training phase. Our work has connections to prequential analysis and intrinsic Bayes factors, but is motivated in a different way. |
| Author | Fong, E Holmes, C C |
| Author_xml | – sequence: 1 givenname: E surname: Fong fullname: Fong, E email: edwin.fong@stats.ox.ac.uk organization: Department of Statistics, University of Oxford, 24–29 St Giles’, Oxford OX1 3LB, UK – sequence: 2 givenname: C C surname: Holmes fullname: Holmes, C C organization: Department of Statistics, University of Oxford, 24–29 St Giles’, Oxford OX1 3LB, UK |
| BookMark | eNotj7FOwzAURS1UJNLCyJ6RxeQlfraTEVVAkSp1gTl6jh1qSO0qDkjw9bQK09VZrs5ZskWIwTF2W8J9CY0ojI8HNxWUfkHrC5aVqJALWcKCZQCguEDEK7ZM6eOMSqqMFbuQT3uXH2h894GGfPCfbvD7GG1OwebdGFPi3zR4S5OP4Zpd9jQkd_O_K_b29Pi63vDt7vll_bDlHYKYuHAkO6OlhRqF1oaMsVq4SpqKDNYnKWqs7KUCK6hCp3vnOrKq7qlrEBuxYnfzb_w6tsfRn_x-2hLac2g7h7ZzqPgDqwZKwQ |
| CitedBy_id | crossref_primary_10_1007_s42113_023_00173_6 crossref_primary_10_1016_j_ejor_2023_04_034 crossref_primary_10_1371_journal_pone_0288000 crossref_primary_10_1093_jrsssb_qkad005 crossref_primary_10_1177_13548166241266912 crossref_primary_10_1007_s11229_021_03233_1 crossref_primary_10_1016_j_sigpro_2024_109735 crossref_primary_10_1111_rssc_12557 crossref_primary_10_1111_insr_12502 crossref_primary_10_1109_TPAMI_2023_3299568 crossref_primary_10_1016_j_dsp_2023_104103 crossref_primary_10_1016_j_eswa_2023_120705 crossref_primary_10_1016_j_gr_2022_07_011 crossref_primary_10_1007_s42113_020_00091_x crossref_primary_10_7717_peerj_cs_904 crossref_primary_10_1093_biomet_asab005 crossref_primary_10_1016_j_jeconom_2023_105491 crossref_primary_10_1137_20M1310849 crossref_primary_10_1371_journal_pcbi_1009070 crossref_primary_10_1016_j_cogpsych_2023_101562 crossref_primary_10_1016_j_mex_2025_103336 crossref_primary_10_1371_journal_pone_0290331 crossref_primary_10_1093_jrsssb_qkaf015 crossref_primary_10_1186_s12711_022_00765_z crossref_primary_10_1016_j_ejor_2022_04_029 crossref_primary_10_1093_jrsssb_qkad094 crossref_primary_10_1002_sta4_600 crossref_primary_10_1038_s41598_022_20872_7 crossref_primary_10_1016_j_compag_2025_110136 crossref_primary_10_1016_j_ress_2024_110094 crossref_primary_10_1111_bmsp_12314 crossref_primary_10_1287_mnsc_2023_4801 crossref_primary_10_1038_s41588_023_01583_9 crossref_primary_10_1007_s42113_022_00158_x crossref_primary_10_1111_rssc_12488 crossref_primary_10_1007_s11222_023_10205_7 crossref_primary_10_1027_2151_2604_a000555 crossref_primary_10_1093_mnras_stac3532 crossref_primary_10_1073_pnas_2401230121 crossref_primary_10_1080_00031305_2023_2216239 crossref_primary_10_1016_j_neuroimage_2021_118780 |
| ContentType | Journal Article |
| Copyright | 2020 Biometrika Trust 2020 |
| Copyright_xml | – notice: 2020 Biometrika Trust 2020 |
| DBID | TOX |
| DOI | 10.1093/biomet/asz077 |
| DatabaseName | Oxford Journals Open Access Collection |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: TOX name: Oxford Journals Open Access Collection url: https://academic.oup.com/journals/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Statistics Biology |
| EISSN | 1464-3510 |
| EndPage | 496 |
| ExternalDocumentID | 10.1093/biomet/asz077 |
| GroupedDBID | -DZ -E4 -~X ..I .2P .DC .I3 0R~ 1TH 23N 3R3 4.4 482 48X 5GY 5RE 5VS 5WA 6J9 6OB 70D 79B AAIJN AAJKP AAMVS AAOGV AAPQZ AAPXW AARHZ AASNB AAUAY AAUQX AAVAP ABDTM ABEUO ABFAN ABIXL ABJNI ABLJU ABNKS ABPFR ABPPZ ABPTD ABQLI ABWST ABXVV ABYWD ABZBJ ACBEA ACGFO ACGFS ACGOD ACIPB ACIWK ACMTB ACNCT ACPRK ACTMH ACUFI ACUTJ ACYTK ADEYI ADEZT ADGZP ADHKW ADHZD ADIPN ADLSF ADOCK ADQBN ADRDM ADRIX ADRTK ADVEK ADYVW ADZXQ AECKG AEGPL AEGXH AEJOX AEKKA AEKSI AEMDU AENEX AENZO AEPUE AETBJ AEUPB AEWNT AFFZL AFIYH AFOFC AFRAH AFVYC AFXEN AFXHP AGINJ AGKEF AGQXC AGSYK AHXPO AIAGR AIJHB AJEEA AJEUX ALMA_UNASSIGNED_HOLDINGS ALTZX ALUQC APIBT APWMN ATGXG AXUDD AZVOD BAYMD BCRHZ BEYMZ BHONS BQUQU BTQHN C45 CDBKE CS3 CZ4 DAKXR DILTD DU5 D~K EBS EE~ ESX F5P F9B FLIZI FLUFQ FOEOM FQBLK GAUVT GJXCC H13 H5~ HAR HW0 HZ~ IOX J21 JAS JXSIZ KAQDR KBUDW KOP KSI KSN M-Z M49 ML0 N9A NGC NMDNZ NOMLY NU- O9- ODMLO OJQWA OJZSN OVD OWPYF P2P PAFKI PEELM PQQKQ Q1. Q5Y R44 RD5 ROL ROX ROZ RUSNO RW1 RXO TEORI TJP TN5 TOX WH7 X7H XSW YAYTL YKOAZ YXANX ZKX ~02 ~91 |
| ID | FETCH-LOGICAL-c403t-3ea5cb75d084377babbd73e25b2ab48351a9d5f560d3a24e7feecad68fac94493 |
| IEDL.DBID | TOX |
| ISICitedReferencesCount | 64 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000558976700018&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0006-3444 |
| IngestDate | Wed Sep 11 04:40:22 EDT 2024 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 2 |
| Keywords | cross-validation Prequential scoring Marginal likelihood |
| Language | English |
| License | This is an Open Access article distributed under the terms of the Creative CommonsAttribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited. |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c403t-3ea5cb75d084377babbd73e25b2ab48351a9d5f560d3a24e7feecad68fac94493 |
| OpenAccessLink | https://dx.doi.org/10.1093/biomet/asz077 |
| PageCount | 8 |
| ParticipantIDs | oup_primary_10_1093_biomet_asz077 |
| PublicationCentury | 2000 |
| PublicationDate | 2020-06-01 |
| PublicationDateYYYYMMDD | 2020-06-01 |
| PublicationDate_xml | – month: 06 year: 2020 text: 2020-06-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationTitle | Biometrika |
| PublicationYear | 2020 |
| Publisher | Oxford University Press |
| Publisher_xml | – name: Oxford University Press |
| SSID | ssj0006656 |
| Score | 2.5969617 |
| Snippet | Summary
In Bayesian statistics, the marginal likelihood, also known as the evidence, is used to evaluate model fit as it quantifies the joint probability of... |
| SourceID | oup |
| SourceType | Publisher |
| StartPage | 489 |
| Title | On the marginal likelihood and cross-validation |
| Volume | 107 |
| WOSCitedRecordID | wos000558976700018&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1JSwMxGA1SFHpxqYo7EbyGTifb5Chi8SCthyq9DV82KNZR2irorzfLIIIeevsOWeAlJC_5-N5D6ErpeOxZTUBKRxgvQ-StIJJWBdOuAEh2b0_3cjSqplP10P53LP9J4SvaT3Xoqz4svwoZy8YHvIobejKe_hy5QiSb1hgRyhhrxTT_9M5VbL8ukOHO-lPvou2WJOLrvKp7aMM1PbSVbSM_e6gbGWIWWN5H_XGDA4nDL7BIDld4Pnt281lUK8bQWJyuQRL20yy7Jx2gx-Ht5OaOtC4IxLCCrgh1wI2W3BYVo1Jq0NpK6kquS9AsEKgBKMt9YC6WQsmc9M4ZsKLyYBRjih6iTvPauCOEPeVSSe4dhSjjJrQxYUQqmOcKqIBjdBngqd-yzkWd89O0zijUGYWTNdqcom4Zn6Xps-IMdVaLd3eONs1HwGZxkZbuG4dQmGc |
| linkProvider | Oxford University Press |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=On+the+marginal+likelihood+and+cross-validation&rft.jtitle=Biometrika&rft.au=Fong%2C+E&rft.au=Holmes%2C+C+C&rft.date=2020-06-01&rft.pub=Oxford+University+Press&rft.issn=0006-3444&rft.eissn=1464-3510&rft.volume=107&rft.issue=2&rft.spage=489&rft.epage=496&rft_id=info:doi/10.1093%2Fbiomet%2Fasz077&rft.externalDocID=10.1093%2Fbiomet%2Fasz077 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0006-3444&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0006-3444&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0006-3444&client=summon |