Convergence of batch gradient algorithm with smoothing composition of group l0 and l1/2 regularization for feedforward neural networks

In this paper, we prove the convergence of batch gradient method for training feedforward neural network; we have proposed a new penalty term based on composition of smoothing L 1 / 2 penalty for weights vectors incoming to hidden nodes and smoothing group L 0 regularization for the resulting vector...

Full description

Saved in:
Bibliographic Details
Published in:Progress in artificial intelligence Vol. 11; no. 3; pp. 269 - 278
Main Authors: Ramchoun, Hassan, Ettaouil, Mohamed
Format: Journal Article
Language:English
Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.09.2022
Springer Nature B.V
Subjects:
Algorithms
Artificial Intelligence
Artificial neural networks
Composition
Computational Intelligence
Computer Imaging
Computer Science
Control
Convergence
Data Mining and Knowledge Discovery
Mechatronics
Natural Language Processing (NLP)
Nodes
Pattern Recognition and Graphics
Regular Paper
Regularization
Robotics
Smoothing
Training
Vision
Multilayer perceptron
Smoothing group
Smoothing
regularization
Convergence
ISSN:2192-6352, 2192-6360
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this paper, we prove the convergence of batch gradient method for training feedforward neural network; we have proposed a new penalty term based on composition of smoothing L 1 / 2 penalty for weights vectors incoming to hidden nodes and smoothing group L 0 regularization for the resulting vector (BGSGL 0 L 1 / 2 ). This procedure forces weights to become smaller in group level, after training, which allow to remove some redundant hidden nodes. Moreover, it can remove some redundant weights of the surviving hidden nodes. The conditions of convergence are given. The importance of our proposed regularization objective is also tested on numerical examples of classification and regression task.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2192-6352
2192-6360
DOI:10.1007/s13748-022-00285-3