Realistic Evaluation of Deep Active Learning for Image Classification and Semantic Segmentation

Active learning aims to reduce the high labeling cost involved in training machine learning models on large datasets by efficiently labeling only the most informative samples. Recently, deep active learning has shown success on various tasks. However, the conventional evaluation schemes are either i...

Full description

Saved in:
Bibliographic Details
Published in:International journal of computer vision Vol. 133; no. 7; pp. 4294 - 4316
Main Authors: Mittal, Sudhanshu, Niemeijer, Joshua, Çiçek, Özgün, Tatarchenko, Maxim, Ehrhardt, Jan, Schäfer, Jörg P., Handels, Heinz, Brox, Thomas
Format: Journal Article
Language:English
Published: New York Springer US 01.07.2025
Springer Nature B.V
Subjects:
Artificial Intelligence
Budgets
Computer Imaging
Computer Science
Data augmentation
Datasets
German Conference on Pattern Recognition special issues (GCPR)
Image annotation
Image classification
Image Processing and Computer Vision
Image segmentation
Labeling
Machine learning
Pattern Recognition
Pattern Recognition and Graphics
Redundancy
Semantic segmentation
Semantics
Semi-supervised learning
Vision
Segmentation
Active learning
Semi supervised learning
Classification
ISSN:0920-5691, 1573-1405
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Active learning aims to reduce the high labeling cost involved in training machine learning models on large datasets by efficiently labeling only the most informative samples. Recently, deep active learning has shown success on various tasks. However, the conventional evaluation schemes are either incomplete or below par. This study critically assesses various active learning approaches, identifying key factors essential for choosing the most effective active learning method. It includes a comprehensive guide to obtain the best performance for each case, in image classification and semantic segmentation. For image classification, the AL methods improve by a large-margin when integrated with data augmentation and semi-supervised learning, but barely perform better than the random baseline. In this work, we evaluate them under more realistic settings and propose a more suitable evaluation protocol. For semantic segmentation, previous academic studies focused on diverse datasets with substantial annotation resources. In contrast, data collected in many driving scenarios is highly redundant, and most medical applications are subject to very constrained annotation budgets. The study evaluates active learning techniques under various conditions including data redundancy, the use of semi-supervised learning, and differing annotation budgets. As an outcome of our study, we provide a comprehensive usage guide to obtain the best performance for each case.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0920-5691
1573-1405
DOI:10.1007/s11263-025-02372-z