Convergence of batch gradient algorithm with smoothing composition of group l0 and l1/2 regularization for feedforward neural networks

In this paper, we prove the convergence of batch gradient method for training feedforward neural network; we have proposed a new penalty term based on composition of smoothing L 1 / 2 penalty for weights vectors incoming to hidden nodes and smoothing group L 0 regularization for the resulting vector...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Progress in artificial intelligence Ročník 11; číslo 3; s. 269 - 278
Hlavní autoři: Ramchoun, Hassan, Ettaouil, Mohamed
Médium: Journal Article
Jazyk:angličtina
Vydáno: Berlin/Heidelberg Springer Berlin Heidelberg 01.09.2022
Springer Nature B.V
Témata:
Algorithms
Artificial Intelligence
Artificial neural networks
Composition
Computational Intelligence
Computer Imaging
Computer Science
Control
Convergence
Data Mining and Knowledge Discovery
Mechatronics
Natural Language Processing (NLP)
Nodes
Pattern Recognition and Graphics
Regular Paper
Regularization
Robotics
Smoothing
Training
Vision
Multilayer perceptron
Smoothing group
Smoothing
regularization
Convergence
ISSN:2192-6352, 2192-6360
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:In this paper, we prove the convergence of batch gradient method for training feedforward neural network; we have proposed a new penalty term based on composition of smoothing L 1 / 2 penalty for weights vectors incoming to hidden nodes and smoothing group L 0 regularization for the resulting vector (BGSGL 0 L 1 / 2 ). This procedure forces weights to become smaller in group level, after training, which allow to remove some redundant hidden nodes. Moreover, it can remove some redundant weights of the surviving hidden nodes. The conditions of convergence are given. The importance of our proposed regularization objective is also tested on numerical examples of classification and regression task.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2192-6352
2192-6360
DOI:10.1007/s13748-022-00285-3