Improvements on Cross-Validation: The 632+ Bootstrap Method

A training set of data has been used to construct a rule for predicting future responses. What is the error rate of this rule? This is an important question both for comparing models and for assessing a final selected model. The traditional answer to this question is given by cross-validation. The c...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Journal of the American Statistical Association Ročník 92; číslo 438; s. 548 - 560
Hlavní autori: Efron, Bradley, Tibshirani, Robert
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Alexandria, VA Taylor & Francis Group 01.06.1997
American Statistical Association
Predmet:
Bootstrap resampling
Classification
Cross-validation bootstrap
Error rates
Estimation bias
Estimation methods
Estimators
Exact sciences and technology
Mathematics
Multivariate analysis
Nonparametric inference
Point estimators
Prediction rule
Probability and statistics
Sciences and techniques of general use
Simulation training
Standard deviation
Standard error
Statistical methods
Statistical variance
Statistics
Theory and Methods
Error estimation
Error prediction
Asymptotic behavior
Prediction
Samplings
Error rate
Non parametric method
Cross validation
Discriminant function
Prediction rule
Classification
Classification problem
Bootstrap
Numerical simulation
Fisher function
ISSN:0162-1459, 1537-274X
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:A training set of data has been used to construct a rule for predicting future responses. What is the error rate of this rule? This is an important question both for comparing models and for assessing a final selected model. The traditional answer to this question is given by cross-validation. The cross-validation estimate of prediction error is nearly unbiased but can be highly variable. Here we discuss bootstrap estimates of prediction error, which can be thought of as smoothed versions of cross-validation. We show that a particular bootstrap method, the .632+ rule, substantially outperforms cross-validation in a catalog of 24 simulation experiments. Besides providing point estimates, we also consider estimating the variability of an error rate estimate. All of the results here are nonparametric and apply to any possible prediction rule; however, we study only classification problems with 0-1 loss in detail. Our simulations include "smooth" prediction rules like Fisher's linear discriminant function and unsmooth ones like nearest neighbors.
Bibliografia:ObjectType-Article-2
SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 23
ISSN:0162-1459
1537-274X
DOI:10.1080/01621459.1997.10474007