Benchmarking AutoML frameworks for disease prediction using medical claims

Objectives Ascertain and compare the performances of Automated Machine Learning (AutoML) tools on large, highly imbalanced healthcare datasets. Materials and Methods We generated a large dataset using historical de-identified administrative claims including demographic information and flags for dise...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:BioData mining Ročník 15; číslo 1; s. 1 - 13
Hlavní autori: A. Romero, Roland Albert, Y. Deypalan, Mariefel Nicole, Mehrotra, Suchit, Jungao, John Titus, Sheils, Natalie E., Manduchi, Elisabetta, Moore, Jason H.
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: London BioMed Central 26.07.2022
BioMed Central Ltd
Springer Nature B.V
BMC
Predmet:
Algorithms
Analysis
Automated machine learning
AutoML
Benchmarks
Big Data
Bioinformatics
Biomedical and Life Sciences
Class imbalance
Codes
Computational Biology/Bioinformatics
Computer Appl. in Life Sciences
Data Mining and Knowledge Discovery
Datasets
Diabetes
Disease
Health care
Healthcare
Life Sciences
Lung cancer
Machine learning
Medical claims
Medical research
Medicine, Experimental
Performance prediction
Population
Resampling
Short Report
Subject specialists
Automated machine learning
Healthcare
AutoML
Medical claims
Class imbalance
Machine learning
ISSN:1756-0381, 1756-0381
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Objectives Ascertain and compare the performances of Automated Machine Learning (AutoML) tools on large, highly imbalanced healthcare datasets. Materials and Methods We generated a large dataset using historical de-identified administrative claims including demographic information and flags for disease codes in four different time windows prior to 2019. We then trained three AutoML tools on this dataset to predict six different disease outcomes in 2019 and evaluated model performances on several metrics. Results The AutoML tools showed improvement from the baseline random forest model but did not differ significantly from each other. All models recorded low area under the precision-recall curve and failed to predict true positives while keeping the true negative rate high. Model performance was not directly related to prevalence. We provide a specific use-case to illustrate how to select a threshold that gives the best balance between true and false positive rates, as this is an important consideration in medical applications. Discussion Healthcare datasets present several challenges for AutoML tools, including large sample size, high imbalance, and limitations in the available features. Improvements in scalability, combinations of imbalance-learning resampling and ensemble approaches, and curated feature selection are possible next steps to achieve better performance. Conclusion Among the three explored, no AutoML tool consistently outperforms the rest in terms of predictive performance. The performances of the models in this study suggest that there may be room for improvement in handling medical claims data. Finally, selection of the optimal prediction threshold should be guided by the specific practical application.
Bibliografia:SourceType-Scholarly Journals-1
content type line 14
ObjectType-Report-1
ObjectType-Article-1
ObjectType-Feature-2
content type line 23
ISSN:1756-0381
1756-0381
DOI:10.1186/s13040-022-00300-2