GPU‐accelerated and mixed norm regularized online extreme learning machine

Summary Extreme learning machine (ELM) is a prominent example of neural network with its fast training speed, and good prediction performance. An online version of ELM called online sequential extreme learning machine (OS‐ELM) has also been proposed for the sequential training. Combined with the nee...

Full description

Saved in:
Bibliographic Details
Published in:Concurrency and computation Vol. 34; no. 15
Main Authors: Polat, Önder, Kayhan, Sema Koç
Format: Journal Article
Language:English
Published: Hoboken Wiley Subscription Services, Inc 10.07.2022
Subjects:
Acceleration
Algorithms
alternating direction method of multipliers
Artificial neural networks
extreme learning machine
graphics processing unit
Graphics processing units
Machine learning
Neural networks
online sequential learning
Parameters
Regularization
Training
Tuning
ISSN:1532-0626, 1532-0634
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Summary Extreme learning machine (ELM) is a prominent example of neural network with its fast training speed, and good prediction performance. An online version of ELM called online sequential extreme learning machine (OS‐ELM) has also been proposed for the sequential training. Combined with the need for regularization to prevent over‐fitting in addition to the large number of neurons required in the hidden layer, OS‐ELM demands huge amount of computation power for the large‐scale data. In this article, a mixed norm (l2,1$$ {l}_{2,1} $$) regularized online machine learning algorithm (MRO‐ELM) that is based on alternating direction method of multipliers (ADMM) is proposed. A linear combination of the mixed norm and the Frobenius norm regularization is applied using the ADMM framework and update formulas are derived. Graphics processing unit (GPU) accelerated version of MRO‐ELM (GPU‐MRO‐ELM) is also proposed to reduce the training time by processing appropriate parts in parallel using the implemented custom kernels. In addition, a novel automatic hyper‐parameter tuning method is incorporated to GPU‐MRO‐ELM using progressive validation with GPU acceleration. The experimental results show that the MRO‐ELM algorithm and its GPU version outperform OS‐ELM in terms of training speed, and testing accuracy. Also, compared to the cross validation, the proposed automatic hyper‐parameter tuning demonstrates dramatical reduction in the tuning time.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1532-0626
1532-0634
DOI:10.1002/cpe.6967