Ensuring medical AI safety: interpretability-driven detection and mitigation of spurious model behavior and associated data

Deep neural networks are increasingly employed in high-stakes medical applications, despite their tendency for shortcut learning in the presence of spurious correlations, which can have potentially fatal consequences in practice. Whereas a multitude of works address either the detection or mitigatio...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Machine learning Jg. 114; H. 9; S. 206
Hauptverfasser: Pahde, Frederik, Wiegand, Thomas, Lapuschkin, Sebastian, Samek, Wojciech
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York Springer US 01.09.2025
Springer Nature B.V
Schlagworte:
Annotations
Artifact identification
Artificial Intelligence
Artificial neural networks
Automation
Bias
Computer Science
Control
Datasets
Identification
Localization
Machine Learning
Mechatronics
Medical equipment
Melanoma
Natural Language Processing (NLP)
Neural networks
Robotics
Simulation and Modeling
Subject specialists
Explainable artificial intelligence
Bias mitigation
Data annotation
Interpretability
Spurious correlations
ISSN:0885-6125, 1573-0565
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Deep neural networks are increasingly employed in high-stakes medical applications, despite their tendency for shortcut learning in the presence of spurious correlations, which can have potentially fatal consequences in practice. Whereas a multitude of works address either the detection or mitigation of such shortcut behavior in isolation, the Reveal2Revise approach provides a comprehensive bias mitigation framework combining these steps. However, effectively addressing these biases often requires substantial labeling efforts from domain experts. In this work, we review the steps of the Reveal2Revise framework and enhance it with semi-automated interpretability-based bias annotation capabilities. This includes methods for the sample- and feature-level bias annotation, providing valuable information for bias mitigation methods to unlearn the undesired shortcut behavior. We show the applicability of the framework using four medical datasets across two modalities, featuring controlled and real-world spurious correlations caused by data artifacts. We successfully identify and mitigate these biases in VGG16, ResNet50, and contemporary Vision Transformer models, ultimately increasing their robustness and applicability for real-world medical tasks. Our code is available at https://github.com/frederikpahde/medical-ai-safety .
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
Editors: Annalisa Appice, Giuseppina Andresini, Przemyslaw Biecek, Christian Wressnegger.
ISSN:0885-6125
1573-0565
DOI:10.1007/s10994-025-06834-w