Improvements on Cross-Validation: The 632+ Bootstrap Method

A training set of data has been used to construct a rule for predicting future responses. What is the error rate of this rule? This is an important question both for comparing models and for assessing a final selected model. The traditional answer to this question is given by cross-validation. The c...

Full description

Saved in:

Bibliographic Details
Published in:	Journal of the American Statistical Association Vol. 92; no. 438; pp. 548 - 560
Main Authors:	Efron, Bradley, Tibshirani, Robert
Format:	Journal Article
Language:	English
Published:	Alexandria, VA Taylor & Francis Group 01.06.1997 American Statistical Association
Subjects:	Bootstrap resampling Classification Cross-validation bootstrap Error rates Estimation bias Estimation methods Estimators Exact sciences and technology Mathematics Multivariate analysis Nonparametric inference Point estimators Prediction rule Probability and statistics Sciences and techniques of general use Simulation training Standard deviation Standard error Statistical methods Statistical variance Statistics Theory and Methods Error estimation Error prediction Asymptotic behavior Prediction Samplings Error rate Non parametric method Cross validation Discriminant function Prediction rule Classification Classification problem Bootstrap Numerical simulation Fisher function
ISSN:	0162-1459, 1537-274X
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Abstract	A training set of data has been used to construct a rule for predicting future responses. What is the error rate of this rule? This is an important question both for comparing models and for assessing a final selected model. The traditional answer to this question is given by cross-validation. The cross-validation estimate of prediction error is nearly unbiased but can be highly variable. Here we discuss bootstrap estimates of prediction error, which can be thought of as smoothed versions of cross-validation. We show that a particular bootstrap method, the .632+ rule, substantially outperforms cross-validation in a catalog of 24 simulation experiments. Besides providing point estimates, we also consider estimating the variability of an error rate estimate. All of the results here are nonparametric and apply to any possible prediction rule; however, we study only classification problems with 0-1 loss in detail. Our simulations include "smooth" prediction rules like Fisher's linear discriminant function and unsmooth ones like nearest neighbors.
AbstractList	A training set of data has been used to construct a rule for predicting future responses. What is the error rate of this rule? This is an important question both for comparing models and for assessing a final selected model. The traditional answer to this question is given by cross-validation. The cross-validation estimate of prediction error is nearly unbiased but can be highly variable. Here we discuss bootstrap estimates of prediction error, which can be thought of as smoothed versions of cross-validation. We show that a particular bootstrap method, the .632+ rule, substantially outperforms cross-validation in a catalog of 24 simulation experiments. Besides providing point estimates, we also consider estimating the variability of an error rate estimate. All of the results here are nonparametric and apply to any possible prediction rule; however, we study only classification problems with 0-1 loss in detail. Our simulations include "smooth" prediction rules like Fisher's linear discriminant function and unsmooth ones like nearest neighbors.
Author	Tibshirani, Robert Efron, Bradley
Author_xml	– sequence: 1 givenname: Bradley surname: Efron fullname: Efron, Bradley organization: Department of Statistics , Stanford University – sequence: 2 givenname: Robert surname: Tibshirani fullname: Tibshirani, Robert organization: Department of Preventive Medicine and Biostatistics , University of Toronto
BackLink	http://pascal-francis.inist.fr/vibad/index.php?action=getRecordDetail&idt=2812655$$DView record in Pascal Francis
BookMark	eNqFkEtLAzEYRYMoWKt_QWYhbmQ0j0kyUTe1-ChU3Ki4C2kmwSkzk5qkiv_ejG1B3DSbQDg333fuAdjtXGcAOEbwHMESXkDEMCqoOEdC8PRU8AJCvgMGiBKeY1687YJBD-U9tQ8OQpjDdHhZDsDVpF1492la08WQuS4bexdC_qqaulKxdt1l9vxuMkbwWXbjXAzRq0X2aOK7qw7BnlVNMEfrewhe7m6fxw_59Ol-Mh5Nc00QjXkhDGaWiQoqrrlAs5JAboQVpYXMUouxsjNbWA1nhGGibKVNOasoZVzjooJkCE5X_6ZNP5YmRNnWQZumUZ1xyyBJSTlknCXwZA2qoFVjvep0HeTC163y3xKXCDNKE3a9wnTv6o2Vuo6_skmubiSCsi9WboqVfbFyU2yKs3_xzYCtwfV68xCd_5vCqRGJBUseJGGjFVZ31vlWfTnfVDKq78b5jRLZMuoHHHqb4Q
CODEN	JSTNAL
CitedBy_id	crossref_primary_10_1080_14036096_2022_2149618 crossref_primary_10_32890_jict_20_1_2021_6358 crossref_primary_10_1080_01431161_2010_486413 crossref_primary_10_1080_19466315_2020_1835710 crossref_primary_10_1108_17563780910959901 crossref_primary_10_3389_fgene_2019_00724 crossref_primary_10_3389_fpsyt_2022_828773 crossref_primary_10_2217_14622416_7_7_1003 crossref_primary_10_3389_fimmu_2019_03109 crossref_primary_10_1080_10543406_2011_629890 crossref_primary_10_1063_5_0281324 crossref_primary_10_1007_s10750_024_05675_3 crossref_primary_10_2196_28333 crossref_primary_10_1002_nop2_1602 crossref_primary_10_1007_s13595_021_01047_2 crossref_primary_10_1214_009053604000000959 crossref_primary_10_1080_13658816_2013_871016 crossref_primary_10_1080_02664763_2023_2252208 crossref_primary_10_1164_rccm_200612_1819OC crossref_primary_10_1007_s10096_016_2714_9 crossref_primary_10_1007_s00442_018_4086_6 crossref_primary_10_1080_00031305_2015_1005128 crossref_primary_10_1109_TMI_2012_2206398 crossref_primary_10_3389_fgene_2021_651971 crossref_primary_10_1111_1752_1688_13250 crossref_primary_10_1186_s12916_023_02858_y crossref_primary_10_1029_2024JD041671 crossref_primary_10_5194_essd_13_1829_2021 crossref_primary_10_2217_14622416_7_4_543 crossref_primary_10_1007_s11294_024_09898_x crossref_primary_10_1128_JCM_03467_14 crossref_primary_10_1080_00949655_2020_1856848 crossref_primary_10_1080_02635143_2025_2491073 crossref_primary_10_1080_1062936X_2015_1084647 crossref_primary_10_1080_10543406_2016_1226329 crossref_primary_10_5194_amt_12_525_2019 crossref_primary_10_1093_aje_kwj235 crossref_primary_10_1081_BIP_200025645 crossref_primary_10_1186_s13195_023_01367_7 crossref_primary_10_3389_fgene_2021_749573 crossref_primary_10_1080_00222216_2011_11950235 crossref_primary_10_1093_molbev_msae238 crossref_primary_10_1117_1_JMI_12_S2_S22012 crossref_primary_10_1080_17538947_2025_2520471 crossref_primary_10_1029_2021WR029680 crossref_primary_10_1080_10503307_2020_1859638 crossref_primary_10_3389_fmars_2023_1116829 crossref_primary_10_2196_49023 crossref_primary_10_3389_fbioe_2019_00226 crossref_primary_10_3389_ffgc_2024_1472699 crossref_primary_10_1080_10543406_2010_504906 crossref_primary_10_2139_ssrn_2562051 crossref_primary_10_5194_nhess_21_2993_2021 crossref_primary_10_1080_15592294_2015_1137414 crossref_primary_10_3389_fpsyg_2022_821897 crossref_primary_10_1007_s42947_021_00013_5 crossref_primary_10_1080_03610918_2014_936464 crossref_primary_10_1080_0305215X_2020_1861262 crossref_primary_10_3389_fdata_2020_00015 crossref_primary_10_1158_1078_0432_CCR_07_4534 crossref_primary_10_1038_s41598_017_17701_7 crossref_primary_10_1016_j_eswa_2024_123372 crossref_primary_10_5964_qcmb_14891 crossref_primary_10_2196_17516 crossref_primary_10_1080_10485252_2017_1404598 crossref_primary_10_1214_ss_1063994968 crossref_primary_10_1088_1361_6560_abb0c7 crossref_primary_10_1161_HYPERTENSIONAHA_121_18794 crossref_primary_10_1016_S0140_6736_05_66539_7 crossref_primary_10_3389_fonc_2021_644045 crossref_primary_10_1080_13668803_2024_2413006 crossref_primary_10_1080_10543406_2019_1579220 crossref_primary_10_1080_02724634_2025_2493157 crossref_primary_10_2478_bsrj_2021_0015 crossref_primary_10_1177_09670335231173139 crossref_primary_10_1080_02664763_2022_2070137 crossref_primary_10_1021_acs_jproteome_5c00483 crossref_primary_10_1007_s10651_025_00644_y crossref_primary_10_1029_2020GL091933 crossref_primary_10_1080_02664763_2020_1753023 crossref_primary_10_1080_02664763_2020_1830954 crossref_primary_10_1088_1741_4326_ac1a0d crossref_primary_10_1080_19466315_2020_1828159 crossref_primary_10_5721_EuJRS20154814 crossref_primary_10_1080_07421222_2019_1598692 crossref_primary_10_1007_s00134_017_4678_3 crossref_primary_10_3109_03014460_2012_681800 crossref_primary_10_1080_00015385_2017_1420438 crossref_primary_10_1038_srep35110 crossref_primary_10_1080_01621459_2014_932172 crossref_primary_10_1080_24725854_2017_1347984 crossref_primary_10_1155_2019_8321942 crossref_primary_10_1007_s11205_018_02056_x crossref_primary_10_1080_10503307_2020_1823029 crossref_primary_10_1080_00223131_2018_1467287 crossref_primary_10_1214_009053604000000067 crossref_primary_10_5194_acp_21_35_2021 crossref_primary_10_1093_bioinformatics_bti499 crossref_primary_10_3389_fmars_2019_00645 crossref_primary_10_1080_10618600_2022_2107530 crossref_primary_10_5194_hess_13_1907_2009
Cites_doi	10.1080/00401706.1974.10489157 10.1080/01621459.1986.10478291 10.2307/2336519 10.1080/01621459.1975.10479865 10.1002/0471725293 10.1109/TPAMI.1987.4767957 10.1016/0167-8655(86)90035-8 10.1214/aos/1176344552 10.1016/0167-8655(85)90049-2 10.2307/1403680 10.1080/01621459.1983.10477973 10.1214/aos/1176349027 10.1007/978-1-4899-4541-9
ContentType	Journal Article
Copyright	Copyright Taylor & Francis Group, LLC 1997 Copyright 1997 American Statistical Association 1997 INIST-CNRS
Copyright_xml	– notice: Copyright Taylor & Francis Group, LLC 1997 – notice: Copyright 1997 American Statistical Association – notice: 1997 INIST-CNRS
DBID	AAYXX CITATION IQODW 8BJ FQK JBE
DOI	10.1080/01621459.1997.10474007
DatabaseName	CrossRef Pascal-Francis International Bibliography of the Social Sciences (IBSS) International Bibliography of the Social Sciences International Bibliography of the Social Sciences
DatabaseTitle	CrossRef International Bibliography of the Social Sciences (IBSS)
DatabaseTitleList
DeliveryMethod	fulltext_linktorsrc
Discipline	Statistics Mathematics
EISSN	1537-274X
EndPage	560
ExternalDocumentID	2812655 10_1080_01621459_1997_10474007 2965703 10474007
GroupedDBID	-DZ -~X .-4 ..I .7F .GJ .QJ 0BK 0R~ 1OL 29L 2AX 30N 3R3 4.4 5GY 5RE 692 7WY 7X7 85S 88E 88I 8AF 8C1 8FE 8FG 8FI 8FJ 8FL 8G5 8R4 8R5 AABCJ AAENE AAGDL AAHBH AAHIA AAJMT AALDU AAMIU AAPUL AAQRR AAWIL ABAWQ ABBHK ABCCY ABEFU ABEHJ ABFAN ABFIM ABJCF ABJNI ABLIJ ABLJU ABPAQ ABPEM ABPFR ABPPZ ABPQH ABTAI ABUFD ABUWG ABXSQ ABXUL ABXYU ABYWD ACAGQ ACGFO ACGFS ACGOD ACHJO ACIWK ACMTB ACNCT ACTIO ACTMH ACUBG ADBBV ADCVX ADGTB ADLSF ADMHG ADODI ADULT AEISY AENEX AEPSL AEUPB AEYOC AFFNX AFKRA AFRVT AFVYC AFXHP AGDLA AGLNM AGMYJ AGROQ AHDZW AHMOU AI. AIHAF AIJEM AIYEW AKBVH AKOOK ALCKM ALMA_UNASSIGNED_HOLDINGS ALQZU ALRMG AMEWO AQRUH AQTUD AQUVI AVBZW AWYRJ AZQEC BENPR BEZIV BGLVJ BKNYI BKOMP BLEHA BPHCQ BVXVI CCCUG CCPQU CJ0 CRFIH CS3 D0L DGEBU DKSSO DMQIW DQDLB DSRWC DU5 DWQXO EBS ECEWR EJD E~A E~B F5P FEDTE FJW FRNLG FVMVE FYUFA GNUQQ GROUPED_ABI_INFORM_RESEARCH GTTXZ GUQSH H13 HCIFZ HGD HMCUK HQ6 HVGLF HZ~ H~9 H~P IPSME J.P JAAYA JAS JBMMH JBZCM JENOY JHFFW JKQEH JLEZI JLXEF JMS JPL JST K60 K6~ K9- KYCEM L6V LU7 M0C M0R M0T M1P M2O M2P M4Z M7S MS~ MVM NA5 NHB NUSFT NY~ O9- OK1 P-O P2P PADUT PHGZM PHGZT PJZUB PPXIY PQBIZ PQBZA PQGLB PQQKQ PRG PROAC PSQYO PTHSS Q2X QCRFL RNANH RNS ROSJB RTWRZ RWL RXW S-T S0X SA0 SNACF TAE TASJS TBQAZ TDBHL TEJ TFL TFT TFW TN5 TOXWX TTHFI TUROJ U5U UKHRP UPT UT5 UU3 VH1 VOH WH7 WZA YQT YXB YYM YYP ZGI ZGOLN ~S~ ADYSH AFSUE ALIPV AMPGV 07G AAAVZ AAFWJ AAIKQ AAKBW AAYXX ABRLO ACGEE ACTCW ADXHL AEOZL AEUMN AFFHD AFQQW AGCQS AGLEN AMATQ AMVHM AMXXU BCCOT BPLKW C06 CITATION DWIFK E.L HF~ IPNFZ IVXBP KQ8 LJTGL MW2 OFU RIG SJN TAQ TFMCV UB9 UQL WHG ZCG ZUP ZXP IQODW 8BJ FQK JBE
ID	FETCH-LOGICAL-c315t-49e26f69d0a7c791b8307e9f98f06f5f22afbf4fc0b3623afdce8bd5567c24d03
IEDL.DBID	TFW
ISSN	0162-1459
IngestDate	Tue Oct 21 13:27:50 EDT 2025 Mon Jul 21 09:15:47 EDT 2025 Sat Nov 29 03:56:32 EST 2025 Tue Nov 18 21:37:34 EST 2025 Thu Jun 19 14:56:25 EDT 2025 Mon Oct 20 23:44:37 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	438
Keywords	Error estimation Error prediction Asymptotic behavior Prediction Samplings Error rate Non parametric method Cross validation Discriminant function Prediction rule Classification Classification problem Bootstrap Numerical simulation Fisher function
Language	English
License	CC BY 4.0
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c315t-49e26f69d0a7c791b8307e9f98f06f5f22afbf4fc0b3623afdce8bd5567c24d03
Notes	ObjectType-Article-2 SourceType-Scholarly Journals-1 ObjectType-Feature-1 content type line 23
PQID	38570676
PQPubID	23473
PageCount	13
ParticipantIDs	jstor_primary_10_2307_2965703 crossref_citationtrail_10_1080_01621459_1997_10474007 crossref_primary_10_1080_01621459_1997_10474007 proquest_miscellaneous_38570676 pascalfrancis_primary_2812655 informaworld_taylorfrancis_310_1080_01621459_1997_10474007
PublicationCentury	1900
PublicationDate	19970601
PublicationDateYYYYMMDD	1997-06-01
PublicationDate_xml	– month: 06 year: 1997 text: 19970601 day: 01
PublicationDecade	1990
PublicationPlace	Alexandria, VA
PublicationPlace_xml	– name: Alexandria, VA
PublicationTitle	Journal of the American Statistical Association
PublicationYear	1997
Publisher	Taylor & Francis Group American Statistical Association
Publisher_xml	– name: Taylor & Francis Group – name: American Statistical Association
References	Breiman L. (CIT0002) 1994 Stone M. (CIT0023) 1977; 39 Davison A. C. (CIT0008) 1986; 73 Mallows C. (CIT0020) 1973; 15 Geisser S. (CIT0017) 1975; 70 Breiman L. (CIT0003) 1984 Stone M. (CIT0022) 1974; 36 Jain A. (CIT0018) 1987; 9 Breiman L. (CIT0004) 1992; 60 Chernick M. (CIT0006) 1986; 4 Zhang P. (CIT0025) 1993; 21 Zhang P. (CIT0026) 1995; 22 McLachlan G. (CIT0021) 1992 Efron B. (CIT0013) 1983; 37 Friedman J. (CIT0016) 1994 Efron B. (CIT0011) 1986; 81 Chernick M. (CIT0005) 1985; 4 Efron B. (CIT0015) 1995 Allen D. M. (CIT0001) 1974; 16 Efron B. (CIT0012) 1992 Kohavi R. (CIT0019) 1995 Efron B. (CIT0009) 1979; 7 Efron B. (CIT0010) 1983; 78 Efron B. (CIT0014) 1993 CIT0024 CIT0007
References_xml	– volume: 16 start-page: 125 year: 1974 ident: CIT0001 publication-title: Technometrics doi: 10.1080/00401706.1974.10489157 – start-page: 83 year: 1992 ident: CIT0012 publication-title: Journal of the Royal Statistical Society – volume: 81 start-page: 461 year: 1986 ident: CIT0011 publication-title: Journal of the American Statistical Association doi: 10.1080/01621459.1986.10478291 – volume: 73 start-page: 555 year: 1986 ident: CIT0008 publication-title: Biometrika doi: 10.2307/2336519 – volume: 22 start-page: 83 year: 1995 ident: CIT0026 publication-title: Scandinavian Journal of Statistics – volume: 70 start-page: 320 year: 1975 ident: CIT0017 publication-title: Journal of the American Statistical Association doi: 10.1080/01621459.1975.10479865 – volume: 36 start-page: 111 year: 1974 ident: CIT0022 publication-title: Journal of the Royal Statistical Society – volume-title: Discriminant Analysis and Statistical Pattern Recognition year: 1992 ident: CIT0021 doi: 10.1002/0471725293 – volume: 39 start-page: 44 year: 1977 ident: CIT0023 publication-title: Journal of the Royal Statistical Society – volume: 9 start-page: 628 year: 1987 ident: CIT0018 publication-title: IEEE Transactions on Pattern Analysis and Machine Intelligence doi: 10.1109/TPAMI.1987.4767957 – volume: 37 start-page: 36 year: 1983 ident: CIT0013 publication-title: The American Statistician – volume-title: Classification and Regression Trees year: 1984 ident: CIT0003 – volume: 4 start-page: 133 year: 1986 ident: CIT0006 publication-title: Pattern Recognition Letters doi: 10.1016/0167-8655(86)90035-8 – volume: 7 start-page: 1 year: 1979 ident: CIT0009 publication-title: The Annals of Statistics doi: 10.1214/aos/1176344552 – volume-title: “A Study of Cross-Validation and Bootstrap for Accuracy Assessment and model selection,” year: 1995 ident: CIT0019 – volume: 15 start-page: 661 year: 1973 ident: CIT0020 publication-title: Technometrics – volume-title: “Flexible Metric Nearest-Neighbor Classification,” year: 1994 ident: CIT0016 – volume: 4 start-page: 167 year: 1985 ident: CIT0005 publication-title: Pattern Recognition Letters doi: 10.1016/0167-8655(85)90049-2 – volume-title: “Cross-Validation and the Bootstrap: Estimating the Error Rate of a Prediction Rule,” year: 1995 ident: CIT0015 – ident: CIT0024 – ident: CIT0007 – volume: 60 start-page: 291 year: 1992 ident: CIT0004 publication-title: International Statistical Review doi: 10.2307/1403680 – volume: 78 start-page: 316 year: 1983 ident: CIT0010 publication-title: Journal of the American Statistical Association doi: 10.1080/01621459.1983.10477973 – volume: 21 start-page: 299 year: 1993 ident: CIT0025 publication-title: The Annals of Statistics doi: 10.1214/aos/1176349027 – volume-title: “Bagging Predictors,” year: 1994 ident: CIT0002 – volume-title: An Introduction to the Bootstrap year: 1993 ident: CIT0014 doi: 10.1007/978-1-4899-4541-9
SSID	ssj0000788
Score	2.196224
Snippet	A training set of data has been used to construct a rule for predicting future responses. What is the error rate of this rule? This is an important question...
SourceID	proquest pascalfrancis crossref jstor informaworld
SourceType	Aggregation Database Index Database Enrichment Source Publisher
StartPage	548
SubjectTerms	Bootstrap resampling Classification Cross-validation bootstrap Error rates Estimation bias Estimation methods Estimators Exact sciences and technology Mathematics Multivariate analysis Nonparametric inference Point estimators Prediction rule Probability and statistics Sciences and techniques of general use Simulation training Standard deviation Standard error Statistical methods Statistical variance Statistics Theory and Methods
Title	Improvements on Cross-Validation: The 632+ Bootstrap Method
URI	https://www.tandfonline.com/doi/abs/10.1080/01621459.1997.10474007 https://www.jstor.org/stable/2965703 https://www.proquest.com/docview/38570676
Volume	92
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVPQU databaseName: ABI/INFORM Collection customDbUrl: eissn: 1537-274X dateEnd: 20051231 omitProxy: false ssIdentifier: ssj0000788 issn: 0162-1459 databaseCode: 7WY dateStart: 19920301 isFulltext: true titleUrlDefault: https://www.proquest.com/abicomplete providerName: ProQuest – providerCode: PRVPQU databaseName: ABI/INFORM Global customDbUrl: eissn: 1537-274X dateEnd: 20051231 omitProxy: false ssIdentifier: ssj0000788 issn: 0162-1459 databaseCode: M0C dateStart: 19920301 isFulltext: true titleUrlDefault: https://search.proquest.com/abiglobal providerName: ProQuest – providerCode: PRVPQU databaseName: Consumer Health Database customDbUrl: eissn: 1537-274X dateEnd: 20051231 omitProxy: false ssIdentifier: ssj0000788 issn: 0162-1459 databaseCode: M0R dateStart: 19920301 isFulltext: true titleUrlDefault: https://search.proquest.com/familyhealth providerName: ProQuest – providerCode: PRVPQU databaseName: Engineering Database customDbUrl: eissn: 1537-274X dateEnd: 20051231 omitProxy: false ssIdentifier: ssj0000788 issn: 0162-1459 databaseCode: M7S dateStart: 19920301 isFulltext: true titleUrlDefault: http://search.proquest.com providerName: ProQuest – providerCode: PRVPQU databaseName: Health & Medical Collection customDbUrl: eissn: 1537-274X dateEnd: 20051231 omitProxy: false ssIdentifier: ssj0000788 issn: 0162-1459 databaseCode: 7X7 dateStart: 19920301 isFulltext: true titleUrlDefault: https://search.proquest.com/healthcomplete providerName: ProQuest – providerCode: PRVPQU databaseName: Healthcare Administration Database customDbUrl: eissn: 1537-274X dateEnd: 20051231 omitProxy: false ssIdentifier: ssj0000788 issn: 0162-1459 databaseCode: M0T dateStart: 19920301 isFulltext: true titleUrlDefault: https://search.proquest.com/healthmanagement providerName: ProQuest – providerCode: PRVPQU databaseName: ProQuest Central customDbUrl: eissn: 1537-274X dateEnd: 20051231 omitProxy: false ssIdentifier: ssj0000788 issn: 0162-1459 databaseCode: BENPR dateStart: 19920301 isFulltext: true titleUrlDefault: https://www.proquest.com/central providerName: ProQuest – providerCode: PRVPQU databaseName: Public Health Database customDbUrl: eissn: 1537-274X dateEnd: 20051231 omitProxy: false ssIdentifier: ssj0000788 issn: 0162-1459 databaseCode: 8C1 dateStart: 19920301 isFulltext: true titleUrlDefault: https://search.proquest.com/publichealth providerName: ProQuest – providerCode: PRVPQU databaseName: Research Library customDbUrl: eissn: 1537-274X dateEnd: 20051231 omitProxy: false ssIdentifier: ssj0000788 issn: 0162-1459 databaseCode: M2O dateStart: 19920301 isFulltext: true titleUrlDefault: https://search.proquest.com/pqrl providerName: ProQuest – providerCode: PRVPQU databaseName: Science Database customDbUrl: eissn: 1537-274X dateEnd: 20051231 omitProxy: false ssIdentifier: ssj0000788 issn: 0162-1459 databaseCode: M2P dateStart: 19920301 isFulltext: true titleUrlDefault: https://search.proquest.com/sciencejournals providerName: ProQuest – providerCode: PRVAWR databaseName: Taylor and Francis Online Journals customDbUrl: eissn: 1537-274X dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000788 issn: 0162-1459 databaseCode: TFW dateStart: 19220301 isFulltext: true titleUrlDefault: https://www.tandfonline.com providerName: Taylor & Francis
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1LT9wwEB5R6IFLW_pQl8I2h96qiMSO7ZieyopVLyyIbsv2FNlOfKoSxC79_czYCewKVRzayyhRMnEyHo8_O_MA-KQt54VuTIpgAxcoTIq0LAubulpLj3DfeRGLTajZrFws9HosDLlV0hrax0QRwVbT4DZ2OXjEHSFKofzamiLtFP2kVEUMKMe5n8bmfHr1YIxVKD1JLCnxDEHCf33Mxvy0kb108Fgk90mzRAn6WPrikRUPU9P05f_4qFfwogemydeoSXuw1bSvYZewaEzl_Aa-xA2IsJ-4TLo2mdAXpD8RycfCTMcJKl0iOfucnHTdijZRrpOzUKL6LfyYns4n39K-9kLqeC5WKfYfk17qOjPKKZ3bEo1Bo70ufSa98IwZb33hXWZxCuTG164pbS2EVI4VdcbfwXbbtc17SPARXnICRrUobJNb4-rcFKgHRmnB1AjEIPHK9YnJqT7G7yof8pf2sqlINtUgmxEc3fNdx9QcT3Icr3dotQqbIn03Vvwp5sPQ_ettkRd9xTS5D3G8vqEW9_cxxFBSiBF8HNSkwiFM_2VM23S32DAVGZBK7v_L632AXcRyFJCGhu8Atlc3t80hPHd_UEtuxvBMXf0iulBIy0k-hp2T09nFJZ6dZZNA4_GcKDsP9IKo-j4OQ-oO19wNgw
linkProvider	Taylor & Francis
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1RT9wwDLYQQ4KXARuIG7vRh72himvSJM3taTtxYgJOPByDtyhNmyfUIjj2-2cnLXBCEw_w3DppbcdxHPszwHddcp7r2qbobOABhUmRFkVepq7S0qO777yIzSbUbFZcX-uLFZj0tTCUVklnaB-BIoKtpsVNweg-Je4I3RQC2NZUaqfollLloaL8g8DNlgD059OrJ3OsQvNJokmJqC8T_u84SzvUEn5pn7NICZT2HnnoY_OLF3Y8bE7TzXf5rS342Pmmyc-oTNuwUjefYIPc0Yjm_Bl-xBhECCneJ22TTOgX0j_ozMfeTOME9S6RnB0mv9p2QXGU2-Q8dKnegcvp8XxyknbtF1LHM7FIUYRMeqmrkVVO6aws0B7U2uvCj6QXnjHrS597NypxF-TWV64uykoIqRzLqxHfhdWmbeo9SHAILzn5RpXIyzorrasym6MqWKUFUwMQPcuN67DJqUXGjcl6CNOON4Z4Y3reDODoke42onO8SjF-LlGzCHGRTo6Gv0Y8DPJ_Phcl0humKYOI4_MlvXh8j6EbJYUYwEGvJwZXMV3N2KZuH3Bi6jMglfzyls87gPWT-fmZOfs9O92HjQi0S_Gir7C6uHuoh7Dm_qLG3H0LK-QfThsHyQ
linkToPdf	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1LT8QgECZGjfHi27g-e_BmGrdQoOjJ10ajbjz4uhFKy8m0m3X19zsDre7GGA96bgfamWH4GOZByL7KGUtVaWIAG3BAoYLHWZbmsS2UcAD3reOh2YTs97PnZzWeC4NhlXiGdqFQhLfVuLgHhWsj4g4BpWB9bYWZdhIvKWXqE8pnADsL1PL73tOXNZa-9yTSxEjUZgn_OM7EBjVRvrQNWcT4SfMKLHSh98U3M-73pt7if_zVEllokGl0ElRpmUyV1QqZRzAaajmvkuPggfAOxdeorqIz_IP4EaB86Mx0FIHWRYLRg-i0rkfoRRlEt75H9Rp56F3cn13GTfOF2LKEj2IQIBVOqKJrpJUqyTOwBqVyKnNd4bij1Ljcpc52c9gDmXGFLbO84FxIS9Oiy9bJdFVX5QaJYAgnGCKjgqd5meTGFolJQRGMVJzKDuEtx7VtKpNjg4wXnbQFTBveaOSNbnnTIYefdINQm-NXiqNxgeqR94o0YtTsN-IdL_7xuTCMXlOF8UMMnk-oxed7FECU4LxD9lo10bCG8WLGVGX9BhNjlwEhxeZfPm-PzN2d9_TNVf96i8yHKrvoLNom06PhW7lDZu07KMxw16-PDx4QBns
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Improvements+on+cross-validation%3A+the+.632%2B+bootstrap+method&rft.jtitle=Journal+of+the+American+Statistical+Association&rft.au=Efron%2C+Bradley&rft.au=Tibshirani%2C+Robert&rft.date=1997-06-01&rft.issn=0162-1459&rft.volume=92&rft.issue=438&rft.spage=548&rft.epage=560&rft_id=info:doi/10.1080%2F01621459.1997.10474007&rft.externalDBID=NO_FULL_TEXT
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0162-1459&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0162-1459&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0162-1459&client=summon