Internal-external cross-validation helped to evaluate the generalizability of prediction models in large clustered datasets

To illustrate how to evaluate the need of complex strategies for developing generalizable prediction models in large clustered datasets. We developed eight Cox regression models to estimate the risk of heart failure using a large population-level dataset. These models differed in the number of predi...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	JOURNAL OF CLINICAL EPIDEMIOLOGY Jg. 137; S. 83 - 91
Hauptverfasser:	Takada, Toshihiko, Nijman, Steven, Denaxas, Spiros, Snell, Kym I.E., Uijl, Alicia, Nguyen, Tri-Long, Asselbergs, Folkert W., Debray, Thomas P.A.
Format:	Journal Article Verlag
Sprache:	Englisch
Veröffentlicht:	United States Elsevier Inc 01.09.2021 Elsevier Limited
Schlagworte:	Body mass index Calibration Cluster Analysis Congestive heart failure Datasets Datasets as Topic - statistics & numerical data Discrimination Electronic health records Epidemiology Ethnicity Forecasting Heterogeneity Humans Internal Medicine Maximum likelihood estimation Model comparison Models, Statistical Population Prediction model Prediction models Primary care Regression analysis Regression models Statistical analysis Validation Variables Validation Heterogeneity Model comparison Discrimination Calibration Prediction model
ISSN:	0895-4356, 1878-5921, 1878-5921
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Beschreibung
Zusammenfassung:	To illustrate how to evaluate the need of complex strategies for developing generalizable prediction models in large clustered datasets. We developed eight Cox regression models to estimate the risk of heart failure using a large population-level dataset. These models differed in the number of predictors, the functional form of the predictor effects (non-linear effects and interaction) and the estimation method (maximum likelihood and penalization). Internal-external cross-validation was used to evaluate the models’ generalizability across the included general practices. Among 871,687 individuals from 225 general practices, 43,987 (5.5%) developed heart failure during a median follow-up time of 5.8 years. For discrimination, the simplest prediction model yielded a good concordance statistic, which was not much improved by adopting complex strategies. Between-practice heterogeneity in discrimination was similar in all models. For calibration, the simplest model performed satisfactorily. Although accounting for non-linear effects and interaction slightly improved the calibration slope, it also led to more heterogeneity in the observed/expected ratio. Similar results were found in a second case study involving patients with stroke. In large clustered datasets, prediction model studies may adopt internal-external cross-validation to evaluate the generalizability of competing models, and to identify promising modelling strategies.
Bibliographie:	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 ObjectType-Article-2 ObjectType-Feature-1 content type line 23 ObjectType-Undefined-3
ISSN:	0895-4356 1878-5921 1878-5921
DOI:	10.1016/j.jclinepi.2021.03.025