Identifying daily activities of patient work for type 2 diabetes and co-morbidities: a deep learning and wearable camera approach

People are increasingly encouraged to self-manage their chronic conditions; however, many struggle to practise it effectively. Most studies that investigate patient work (ie, tasks involved in self-management and contexts influencing such tasks) rely on self-reports, which are subject to recall and...

Full description

Saved in:
Bibliographic Details
Published in:Journal of the American Medical Informatics Association : JAMIA Vol. 29; no. 8; p. 1400
Main Authors: Xiong, Hao, Phan, Hoai Nam, Yin, Kathleen, Berkovsky, Shlomo, Jung, Joshua, Lau, Annie Y S
Format: Journal Article
Language:English
Published: England 12.07.2022
Subjects:
Deep Learning
Diabetes Mellitus, Type 2 - therapy
Humans
Morbidity
Neural Networks, Computer
Wearable Electronic Devices
deep learning
patient work
self-management
wearable camera
ISSN:1527-974X, 1527-974X
Online Access:Get more information
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:People are increasingly encouraged to self-manage their chronic conditions; however, many struggle to practise it effectively. Most studies that investigate patient work (ie, tasks involved in self-management and contexts influencing such tasks) rely on self-reports, which are subject to recall and other biases. Few studies use wearable cameras and deep learning to capture and classify patient work activities automatically. We propose a deep learning approach to classify activities of patient work collected from wearable cameras, thereby studying self-management routines more effectively. Twenty-six people with type 2 diabetes and comorbidities wore a wearable camera for a day, generating more than 400 h of video across 12 daily activities. To classify these video images, a weighted ensemble network that combines Linear Discriminant Analysis, Deep Convolutional Neural Networks, and Object Detection algorithms is developed. Performance of our model is assessed using Top-1 and Top-5 metrics, compared against manual classification conducted by 2 independent researchers. Across 12 daily activities, our model achieved on average the best Top-1 and Top-5 scores of 81.9 and 86.8, respectively. Our model also outperformed other non-ensemble techniques in terms of Top-1 and Top-5 scores for most activity classes, demonstrating the superiority of leveraging weighted ensemble techniques. Deep learning can be used to automatically classify daily activities of patient work collected from wearable cameras with high levels of accuracy. Using wearable cameras and a deep learning approach can offer an alternative approach to investigate patient work, one not subjected to biases commonly associated with self-report methods.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:1527-974X
1527-974X
DOI:10.1093/jamia/ocac071