Convergence of a Gradient-Based Learning Algorithm With Penalty for Ridge Polynomial Neural Networks

Recently there have been renewed interests in high order neural networks (HONNs) for its powerful mapping capability. Ridge polynomial neural network (RPNN) is an important kind of HONNs, which always occupies a key position as an efficient instrument in the tasks of classification or regression. In...

Full description

Saved in:
Bibliographic Details
Published in:IEEE access Vol. 9; pp. 28742 - 28752
Main Authors: Fan, Qinwei, Peng, Jigen, Li, Haiyang, Lin, Shoujin
Format: Journal Article
Language:English
Published: Piscataway IEEE 2021
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Algorithms
Biological neural networks
Convergence
high order neural networks
Input variables
Machine learning
Neural networks
Polynomials
Regression analysis
Regularization
ridge polynomial neural network
Smoothing
smoothing Group Lasso
Smoothing methods
Theorems
Training
ISSN:2169-3536, 2169-3536
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Recently there have been renewed interests in high order neural networks (HONNs) for its powerful mapping capability. Ridge polynomial neural network (RPNN) is an important kind of HONNs, which always occupies a key position as an efficient instrument in the tasks of classification or regression. In order to make the convergence speed faster and the network generalization ability stronger, we introduce a regularization model for RPNN with Group Lasso penalty, which deals with the structural sparse problem at the group level in this paper. Nevertheless, there are two main obstacles for introducing the Group Lasso penalty, one is numerical oscillation and the other is convergence analysis challenge. In doing so, we adopt smoothing function to approximate the Group Lasso penalty to overcome these drawbacks. Meanwhile, strong and weak convergence theorems, and monotonicity theorems are provided for this novel algorithm. We also demonstrate the efficiency of our proposed algorithm by numerical experiments, and compare it to the no regularizer, <inline-formula> <tex-math notation="LaTeX">L_{2} </tex-math></inline-formula> regularizer, <inline-formula> <tex-math notation="LaTeX">L_{1/2} </tex-math></inline-formula> regularizer, smoothing <inline-formula> <tex-math notation="LaTeX">L_{1/2} </tex-math></inline-formula> regularizer, and the Group Lasso regularizer, and also the relevant theoretical analysis has been verified.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2020.3048235