A Scalable Algorithm for Active Learning

FIRAL is a recently proposed deterministic active learning algorithm for multiclass classification using logistic regression. It was shown to outperform the state-of-the-art in terms of accuracy and robustness and comes with theoretical performance guarantees. However, its scalability suffers when d...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:SC24: International Conference for High Performance Computing, Networking, Storage and Analysis s. 1 - 17
Hlavní autoři: Chen, Youguang, Wen, Zheyu, Biros, George
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 17.11.2024
Témata:
Accuracy
Active learning
Approximation algorithms
Codes
Computational complexity
contrastive learning
Eigenvalues and eigenfunctions
GPU acceleration
Iterative methods
iterative solvers
message passing interface
performance analysis
randomized linear algebra
Scalability
Synthetic data
Testing
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:FIRAL is a recently proposed deterministic active learning algorithm for multiclass classification using logistic regression. It was shown to outperform the state-of-the-art in terms of accuracy and robustness and comes with theoretical performance guarantees. However, its scalability suffers when dealing with datasets featuring a large number of points n, dimensions d, and classes c, due to its \mathcal{O}\left(c^{2} d^{2}+n c^{2} d\right) storage and \mathcal{O}\left(c^{3}\left(n d^{2}+b d^{3}+b n\right)\right) computational complexity where b is the number of points to select in active learning. To address these challenges, we propose an approximate algorithm with storage requirements reduced to \mathcal{O}\left(n(d+c)+c d^{2}\right) and a computational complexity of \mathcal{O}\left(b n c d^{2}\right). Additionally, we present a parallel implementation on GPUs. We demonstrate the accuracy and scalability of our approach using MNIST, CIFAR-10, Caltech101, and ImageNet. The accuracy tests reveal no deterioration in accuracy compared to FIRAL. We report strong and weak scaling tests on up to 12 GPUs, for three million point synthetic dataset.
DOI:10.1109/SC41406.2024.00057