TransformEHR: transformer-based encoder-decoder generative model to enhance prediction of disease outcomes using electronic health records

Deep learning transformer-based models using longitudinal electronic health records (EHRs) have shown a great success in prediction of clinical diseases or outcomes. Pretraining on a large dataset can help such models map the input space better and boost their performance on relevant tasks through f...

Full description

Saved in:
Bibliographic Details
Published in:Nature communications Vol. 14; no. 1; pp. 7857 - 10
Main Authors: Yang, Zhichao, Mitra, Avijit, Liu, Weisong, Berlowitz, Dan, Yu, Hong
Format: Journal Article
Language:English
Published: London Nature Publishing Group UK 29.11.2023
Nature Publishing Group
Nature Portfolio
Subjects:
639/705/117
692/308/1426
692/700/459
Deep learning
Disease
Electric Power Supplies
Electronic Health Records
Electronic medical records
Encoders-Decoders
Generative artificial intelligence
Humanities and Social Sciences
Humans
Machine learning
Mental disorders
Mental Recall
multidisciplinary
Pancreatic cancer
Pancreatic Neoplasms
Performance prediction
Post traumatic stress disorder
Predictions
Psychological stress
Science
Science (multidisciplinary)
Self destructive behavior
Self-injury
Stress Disorders, Post-Traumatic
System effectiveness
Transformers
ISSN:2041-1723, 2041-1723
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Deep learning transformer-based models using longitudinal electronic health records (EHRs) have shown a great success in prediction of clinical diseases or outcomes. Pretraining on a large dataset can help such models map the input space better and boost their performance on relevant tasks through finetuning with limited data. In this study, we present TransformEHR, a generative encoder-decoder model with transformer that is pretrained using a new pretraining objective—predicting all diseases and outcomes of a patient at a future visit from previous visits. TransformEHR’s encoder-decoder framework, paired with the novel pretraining objective, helps it achieve the new state-of-the-art performance on multiple clinical prediction tasks. Comparing with the previous model, TransformEHR improves area under the precision–recall curve by 2% ( p  < 0.001) for pancreatic cancer onset and by 24% ( p  = 0.007) for intentional self-harm in patients with post-traumatic stress disorder. The high performance in predicting intentional self-harm shows the potential of TransformEHR in building effective clinical intervention systems. TransformEHR is also generalizable and can be easily finetuned for clinical prediction tasks with limited data. Using AI to predict disease can improve interventions slow down or prevent disease. Here, the authors show that generative AI models built on the framework of Transformer, the model that also empowers ChatGPT, can achieve state-of-the-art performance on disease predictions based on longitudinal electronic records.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:2041-1723
2041-1723
DOI:10.1038/s41467-023-43715-z