KAF + RSigELU: a nonlinear and kernel-based activation function for deep neural networks

Activation functions (AFs) are the basis for neural network architectures used in real-world problems to accurately model and learn complex relationships between variables. They are preferred to process the input information coming to the network and to produce the corresponding output. The kernel-b...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Neural computing & applications Ročník 34; číslo 16; s. 13909 - 13923
Hlavní autoři: Kiliçarslan, Serhat, Celik, Mete
Médium: Journal Article
Jazyk:angličtina
Vydáno: London Springer London 01.08.2022
Springer Nature B.V
Témata:
Artificial Intelligence
Artificial neural networks
Complex variables
Computational Biology/Bioinformatics
Computational Science and Engineering
Computer architecture
Computer Science
Data Mining and Knowledge Discovery
Image Processing and Computer Vision
Kernel functions
Machine learning
Neural networks
Original Article
Performance evaluation
Probability and Statistics in Computer Science
Process parameters
Kernel-based activation function (KAF)
CNN
Deep neural network
KAF + RSigELUS
KAF + RSigELUD
ISSN:0941-0643, 1433-3058
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Activation functions (AFs) are the basis for neural network architectures used in real-world problems to accurately model and learn complex relationships between variables. They are preferred to process the input information coming to the network and to produce the corresponding output. The kernel-based activation function (KAF) offers an extended version of ReLU and sigmoid AFs. Therefore, KAF faced with the problems of bias shift originating from the negative region, vanishing gradient, adaptability, flexibility, and neuron death in parameters during the learning process. In this study, hybrid KAF + RSigELUS and KAF + RSigELUD AFs, which are extended versions of KAF, are proposed. In the proposed AFs, the gauss kernel function is used. The proposed KAF + RSigELUS and KAF + RSigELUD AFs are effective in the positive, negative, and linear activation regions. Performance evaluations of them were conducted on the MNIST, Fashion MNIST, CIFAR-10, and SVHN benchmark datasets. The experimental evaluations show that the proposed AFs overcome existing problems and outperformed ReLU, LReLU, ELU, PReLU, and KAF AFs.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0941-0643
1433-3058
DOI:10.1007/s00521-022-07211-7