Identifying Goals of Care Conversations in the Electronic Health Record Using Natural Language Processing and Machine Learning

Goals-of-care discussions are an important quality metric in palliative care. However, goals-of-care discussions are often documented as free text in diverse locations. It is difficult to identify these discussions in the electronic health record (EHR) efficiently. To develop, train, and test an aut...

Full description

Saved in:
Bibliographic Details
Published in:Journal of pain and symptom management Vol. 61; no. 1; pp. 136 - 142.e2
Main Authors: Lee, Robert Y., Brumback, Lyndia C., Lober, William B., Sibley, James, Nielsen, Elizabeth L., Treece, Patsy D., Kross, Erin K., Loggers, Elizabeth T., Fausto, James A., Lindvall, Charlotta, Engelberg, Ruth A., Curtis, J. Randall
Format: Journal Article
Language:English
Published: United States Elsevier Inc 01.01.2021
Elsevier Limited
Subjects:
Anesthesia
Computerized medical records
Documentation
electronic health record
Electronic Health Records
goals of care
Health
Health records
Health services
Humans
Inpatient care
Language acquisition
Learning environment
Machine Learning
medical informatics
Medical records
Natural Language Processing
Objectives
Pain Medicine
Palliative Care
Patient Care Planning
Patients
quality improvement
Quality of care
Subsets
goals of care
Natural language processing
medical informatics
machine learning
electronic health record
quality improvement
ISSN:0885-3924, 1873-6513, 1873-6513
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Goals-of-care discussions are an important quality metric in palliative care. However, goals-of-care discussions are often documented as free text in diverse locations. It is difficult to identify these discussions in the electronic health record (EHR) efficiently. To develop, train, and test an automated approach to identifying goals-of-care discussions in the EHR, using natural language processing (NLP) and machine learning (ML). From the electronic health records of an academic health system, we collected a purposive sample of 3183 EHR notes (1435 inpatient notes and 1748 outpatient notes) from 1426 patients with serious illness over 2008–2016, and manually reviewed each note for documentation of goals-of-care discussions. Separately, we developed a program to identify notes containing documentation of goals-of-care discussions using NLP and supervised ML. We estimated the performance characteristics of the NLP/ML program across 100 pairs of randomly partitioned training and test sets. We repeated these methods for inpatient-only and outpatient-only subsets. Of 3183 notes, 689 contained documentation of goals-of-care discussions. The mean sensitivity of the NLP/ML program was 82.3% (SD 3.2%), and the mean specificity was 97.4% (SD 0.7%). NLP/ML results had a median positive likelihood ratio of 32.2 (IQR 27.5–39.2) and a median negative likelihood ratio of 0.18 (IQR 0.16–0.20). Performance was better in inpatient-only samples than outpatient-only samples. Using NLP and ML techniques, we developed a novel approach to identifying goals-of-care discussions in the EHR. NLP and ML represent a potential approach toward measuring goals-of-care discussions as a research outcome and quality metric.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:0885-3924
1873-6513
1873-6513
DOI:10.1016/j.jpainsymman.2020.08.024