Calibration: the Achilles heel of predictive analytics

Background The assessment of calibration performance of risk prediction models based on regression or more flexible machine learning algorithms receives little attention. Main text Herein, we argue that this needs to change immediately because poorly calibrated algorithms can be misleading and poten...

Full description

Saved in:
Bibliographic Details
Published in:BMC medicine Vol. 17; no. 1; pp. 230 - 7
Main Authors: Van Calster, Ben, McLernon, David J., van Smeden, Maarten, Wynants, Laure, Steyerberg, Ewout W.
Format: Journal Article
Language:English
Published: London BioMed Central 16.12.2019
BioMed Central Ltd
Springer Nature B.V
BMC
Subjects:
Adult
Aged
Algorithms
Analysis
Artificial intelligence
Biomedicine
Calibration
Calibration - standards
Data mining
Decision analysis
Decision making
Embryos
Estimates
Health care policy
Health counseling
Heterogeneity
Humans
In vitro fertilization
Learning algorithms
Machine learning
Machine Learning - standards
Male
Medicine
Medicine & Public Health
Middle Aged
Model performance
Opinion
Overfitting
Patients
Prediction models
Predictive analytics
Predictive Value of Tests
Regression analysis
Risk prediction models
Shared decision making
Heterogeneity
Risk prediction models
Model performance
Calibration
Predictive analytics
Overfitting
ISSN:1741-7015, 1741-7015
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Background The assessment of calibration performance of risk prediction models based on regression or more flexible machine learning algorithms receives little attention. Main text Herein, we argue that this needs to change immediately because poorly calibrated algorithms can be misleading and potentially harmful for clinical decision-making. We summarize how to avoid poor calibration at algorithm development and how to assess calibration at algorithm validation, emphasizing balance between model complexity and the available sample size. At external validation, calibration curves require sufficiently large samples. Algorithm updating should be considered for appropriate support of clinical practice. Conclusion Efforts are required to avoid poor calibration when developing prediction models, to evaluate calibration when validating models, and to update models when indicated. The ultimate aim is to optimize the utility of predictive analytics for shared decision-making and patient counseling.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:1741-7015
1741-7015
DOI:10.1186/s12916-019-1466-7