Calibration and Uncertainty for multiRater Volume Assessment in multiorgan Segmentation (CURVAS) challenge results

Deep learning (DL) has become the dominant approach for medical image segmentation, yet ensuring the reliability and clinical applicability of these models requires addressing key challenges such as annotation variability, calibration, and uncertainty estimation. This is why we created the Calibrati...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Computers in biology and medicine Ročník 197; číslo Pt B; s. 111024
Hlavní autoři: Riera-Marín, Meritxell, O.K., Sikha, Rodríguez-Comas, Júlia, May, Matthias Stefan, Pan, Zhaohong, Zhou, Xiang, Liang, Xiaokun, Erick, Franciskus Xaverius, Prenner, Andrea, Hémon, Cédric, Boussot, Valentin, Dillenseger, Jean-Louis, Nunes, Jean-Claude, Qayyum, Abdul, Mazher, Moona, Niederer, Steven A., Kushibar, Kaisar, Martín-Isla, Carlos, Radeva, Petia, Lekadir, Karim, Barfoot, Theodore, Garcia Peraza Herrera, Luis C., Glocker, Ben, Vercauteren, Tom, Gago, Lucas, Englemann, Justin, Kleiss, Joy-Marie, Aubanell, Anton, Antolin, Andreu, García-López, Javier, González Ballester, Miguel A., Galdrán, Adrián
Médium: Journal Article
Jazyk:angličtina
Vydáno: United States Elsevier Ltd 01.10.2025
Elsevier
Témata:
Abdominal CT
Artificial Intelligence
Bioengineering
Calibration
Computer Science
Deep Learning
Engineering Sciences
Humans
Image Processing, Computer-Assisted - methods
Imaging
Internal Medicine
Life Sciences
Medical Imaging
Multi-class image segmentation
Multiple expert annotations
Other
Signal and Image processing
Uncertainty
Multiple expert annotations
Uncertainty
Multi-class image segmentation
Calibration
Abdominal CT
abdominal CT
ISSN:0010-4825, 1879-0534, 1879-0534
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Deep learning (DL) has become the dominant approach for medical image segmentation, yet ensuring the reliability and clinical applicability of these models requires addressing key challenges such as annotation variability, calibration, and uncertainty estimation. This is why we created the Calibration and Uncertainty for multiRater Volume Assessment in multiorgan Segmentation (CURVAS), which highlights the critical role of multiple annotators in establishing a more comprehensive ground truth, emphasizing that segmentation is inherently subjective and that leveraging inter-annotator variability is essential for robust model evaluation. Seven teams participated in the challenge, submitting a variety of DL models evaluated using metrics such as Dice Similarity Coefficient (DSC), Expected Calibration Error (ECE), and Continuous Ranked Probability Score (CRPS). By incorporating consensus and dissensus ground truth, we assess how DL models handle uncertainty and whether their confidence estimates align with true segmentation performance. Our findings reinforce the importance of well-calibrated models, as better calibration is strongly correlated with the quality of the results. Furthermore, we demonstrate that segmentation models trained on diverse datasets and enriched with pre-trained knowledge exhibit greater robustness, particularly in cases deviating from standard anatomical structures. Notably, the best-performing models achieved high DSC and well-calibrated uncertainty estimates. This work underscores the need for multi-annotator ground truth, thorough calibration assessments, and uncertainty-aware evaluations to develop trustworthy and clinically reliable DL-based medical image segmentation models. •The CURVAS challenge assessed DL segmentation under annotation uncertainty.•Multi-rater CT annotations evaluated consensus and disagreement in three organs.•Metrics account for expert variability, enabling more reliable model assessment.•Calibration-aware models showed strong links between confidence and accuracy.•Volume estimation and robustness to ambiguity support clinical applicability.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:0010-4825
1879-0534
1879-0534
DOI:10.1016/j.compbiomed.2025.111024