Towards human-AI collaboration in radiology: a multidimensional evaluation of the acceptability of AI for chest radiograph analysis in supporting pulmonary tuberculosis diagnosis

Objective Artificial intelligence (AI) technology promises to be a powerful tool in addressing the global health challenges posed by tuberculosis (TB). However, evidence for its real-world impact is lacking, which may hinder safe, responsible adoption. This case study addresses this gap by assessing...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:JAMIA open Jg. 8; H. 1; S. ooae151
Hauptverfasser: Hua, David, Petrina, Neysa, Sacks, Alan J, Young, Noel, Cho, Jin-Gun, Smith, Ross, Poon, Simon K
Format: Journal Article
Sprache:Englisch
Veröffentlicht: United States Oxford University Press 01.02.2025
Schlagworte:
Accuracy
Artificial intelligence
Case studies
Collaboration
Human performance
Medical imaging equipment
Public health
Pulmonary tuberculosis
Research and Applications
Tuberculosis
User experience
Australia
pulmonary tuberculosis
evaluation
translational impact
human factors
artificial intelligence
technical performance
ISSN:2574-2531, 2574-2531
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Objective Artificial intelligence (AI) technology promises to be a powerful tool in addressing the global health challenges posed by tuberculosis (TB). However, evidence for its real-world impact is lacking, which may hinder safe, responsible adoption. This case study addresses this gap by assessing the technical performance, usability and workflow aspects, and health impact of implementing a commercial AI system (qXR by Qure.ai) to support Australian radiologists in diagnosing pulmonary TB. Materials and Methods A retrospective diagnostic accuracy evaluation was conducted to establish the technical performance of qXR in detecting TB compared to a human radiologist and microbiological reference standard. A qualitative human factors assessment was performed to investigate the user experience and clinical decision-making process of radiologists using qXR. A task productivity analysis was completed to quantify how the radiological screening turnaround time is impacted. Results qXR displays near-human performance satisfying the World Health Organization’s suggested accuracy profile. Radiologists reported high satisfaction with using qXR based on minimal workflow disruptions, respect for their professional autonomy, and limited increases in workload burden despite poor algorithm explainability. qXR delivers considerable productivity gains for normal cases and optimizes resource allocation through redistributing time from normal to abnormal cases. Discussion and Conclusion This study provides preliminary evidence of how an AI system with reasonable diagnostic accuracy and a human-centered user experience can meaningfully augment the TB diagnostic workflow. Future research needs to investigate the impact of AI on clinician accuracy, its relationship with efficiency, and best practices for optimizing the impact of clinician-AI collaboration. Lay Summary Artificial intelligence (AI) technology has the potential to transform radiological practice and increase clinician accuracy and efficiency through its ability to triage patient cases and generate diagnostic recommendations. However, evidence for the impact of AI in real-world settings is limited as past studies have focused on its technical performance in highly controlled laboratory settings. Limited attention has been given to how it affects clinical decision-making and healthcare delivery outcomes. This study builds the evidence base for the real-world impact of AI by evaluating the diagnostic accuracy, clinical workflow implications, and task productivity consequences of implementing a commercial AI system called qXR for supporting Australian radiologists in diagnosing tuberculosis (TB). The results offer promising preliminary evidence that qXR performs comparably to human radiologists, optimizes resource allocation through redistributing time spent from normal to abnormal cases, and is regarded favorably by clinicians because of its human-centered user experience and minimal workflow disruptions. This research provides medical institutions with a blueprint for assessing the suitability of AI products for use in their TB diagnostic workflows and specific clinical context. This framework can be continually used in clinical AI monitoring systems to enable issue detection, performance maintenance, and long-term safety and quality assurance.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:2574-2531
2574-2531
DOI:10.1093/jamiaopen/ooae151