Investigating for bias in healthcare algorithms: a sex-stratified analysis of supervised machine learning models in liver disease prediction

ObjectivesThe Indian Liver Patient Dataset (ILPD) is used extensively to create algorithms that predict liver disease. Given the existing research describing demographic inequities in liver disease diagnosis and management, these algorithms require scrutiny for potential biases. We address this over...

Full description

Saved in:
Bibliographic Details
Published in:BMJ health & care informatics Vol. 29; no. 1; p. e100457
Main Authors: Straw, Isabel, Wu, Honghan
Format: Journal Article
Language:English
Published: England BMJ Publishing Group Ltd 01.04.2022
BMJ Publishing Group LTD
BMJ Publishing Group
Subjects:
Algorithms
Artificial Intelligence
Bayes Theorem
Bias
Biomarkers
BMJ Health Informatics
Delivery of Health Care
Female
Gender differences
Health Equity
Health informatics
Humans
Liver Diseases
Machine Learning
Male
Original Research
Public health informatics
Supervised Machine Learning
United Kingdom > UK
Public health informatics
BMJ Health Informatics
Health Equity
Artificial intelligence
Machine Learning
ISSN:2632-1009, 2632-1009
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:ObjectivesThe Indian Liver Patient Dataset (ILPD) is used extensively to create algorithms that predict liver disease. Given the existing research describing demographic inequities in liver disease diagnosis and management, these algorithms require scrutiny for potential biases. We address this overlooked issue by investigating ILPD models for sex bias.MethodsFollowing our literature review of ILPD papers, the models reported in existing studies are recreated and then interrogated for bias. We define four experiments, training on sex-unbalanced/balanced data, with and without feature selection. We build random forests (RFs), support vector machines (SVMs), Gaussian Naïve Bayes and logistic regression (LR) classifiers, running experiments 100 times, reporting average results with SD.ResultsWe reproduce published models achieving accuracies of >70% (LR 71.31% (2.37 SD) – SVM 79.40% (2.50 SD)) and demonstrate a previously unobserved performance disparity. Across all classifiers females suffer from a higher false negative rate (FNR). Presently, RF and LR classifiers are reported as the most effective models, yet in our experiments they demonstrate the greatest FNR disparity (RF; −21.02%; LR; −24.07%).DiscussionWe demonstrate a sex disparity that exists in published ILPD classifiers. In practice, the higher FNR for females would manifest as increased rates of missed diagnosis for female patients and a consequent lack of appropriate care. Our study demonstrates that evaluating biases in the initial stages of machine learning can provide insights into inequalities in current clinical practice, reveal pathophysiological differences between the male and females, and can mitigate the digitisation of inequalities into algorithmic systems.ConclusionOur findings are important to medical data scientists, clinicians and policy-makers involved in the implementation medical artificial intelligence systems. An awareness of the potential biases of these systems is essential in preventing the digital exacerbation of healthcare inequalities.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ObjectType-Review-3
content type line 23
ISSN:2632-1009
2632-1009
DOI:10.1136/bmjhci-2021-100457