FR-MIL: Distribution Re-Calibration-Based Multiple Instance Learning With Transformer for Whole Slide Image Classification

In digital pathology, whole slide images (WSI) are crucial for cancer prognostication and treatment planning. WSI classification is generally addressed using multiple instance learning (MIL), alleviating the challenge of processing billions of pixels and curating rich annotations. Though recent MIL...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on medical imaging Vol. 44; no. 1; pp. 409 - 421
Main Authors: Chikontwe, Philip, Kim, Meejeong, Jeong, Jaehoon, Jung Sung, Hyun, Go, Heounjeong, Jeong Nam, Soo, Park, Sang Hyun
Format: Journal Article
Language:English
Published: United States IEEE 01.01.2025
Subjects:
Algorithms
Benchmark testing
Feature extraction
Histopathology
Humans
Image Interpretation, Computer-Assisted - methods
Machine Learning
Medical diagnostic imaging
multiple instance learning
Multiple-Instance Learning Algorithms
Neoplasms - diagnostic imaging
Neoplasms - pathology
Task analysis
Transformers
Uncertainty
Vectors
weakly supervised learning
whole slide images
ISSN:0278-0062, 1558-254X, 1558-254X
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In digital pathology, whole slide images (WSI) are crucial for cancer prognostication and treatment planning. WSI classification is generally addressed using multiple instance learning (MIL), alleviating the challenge of processing billions of pixels and curating rich annotations. Though recent MIL approaches leverage variants of the attention mechanism to learn better representations, they scarcely study the properties of the data distribution itself i.e., different staining and acquisition protocols resulting in intra-patch and inter-slide variations. In this work, we first introduce a distribution re-calibration strategy to shift the feature distribution of a WSI bag (instances) using the statistics of the max-instance (critical) feature. Second, we enforce class (bag) separation via a metric loss assuming that positive bags exhibit larger magnitudes than negatives. We also introduce a generative process leveraging Vector Quantization (VQ) for improved instance discrimination i.e., VQ helps model bag latent factors for improved classification. To model spatial and context information, a position encoding module (PEM) is employed with transformer-based pooling by multi-head self-attention (PMSA). Evaluation of popular WSI benchmark datasets reveals our approach improves over state-of-the-art MIL methods. Further, we validate the general applicability of our method on classic MIL benchmark tasks and for point cloud classification with limited points. https://github.com/PhilipChicco/FRMIL
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:0278-0062
1558-254X
1558-254X
DOI:10.1109/TMI.2024.3446716