Deterministic convergence analysis for regularized long short-term memory and its application to regression and multi-classification problems

Long short-term memory (LSTM) is a recurrent neural network (RNN) framework designed to solve the gradient disappearance and explosion problems of traditional RNNs. In recent years, LSTM has become a state-of-the-art model for solving various machine-learning problems. This paper propose a novel reg...

Full description

Saved in:

Bibliographic Details
Published in:	Engineering applications of artificial intelligence Vol. 133; p. 108444
Main Authors:	Kang, Qian, Yu, Dengxiu, Cheong, Kang Hao, Wang, Zhen
Format:	Journal Article
Language:	English
Published:	Elsevier Ltd 01.07.2024
Subjects:	Batch gradient algorithm Convergence Long short-term memory Regularization Long short-term memory Batch gradient algorithm Regularization Convergence
ISSN:	0952-1976, 1873-6769
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Description
Summary:	Long short-term memory (LSTM) is a recurrent neural network (RNN) framework designed to solve the gradient disappearance and explosion problems of traditional RNNs. In recent years, LSTM has become a state-of-the-art model for solving various machine-learning problems. This paper propose a novel regularized LSTM based on the batch gradient method. Specifically, the L2 regularization is appended to the objective function as a systematic external force, effectively controlling the excessive growth of weights in the network and preventing the overfitting phenomenon. In addition, a rigorous convergence analysis of the proposed method is carried out, i.e., monotonicity, weak convergence, and strong convergence results are obtained. Finally, comparative simulations are conducted on the benchmark data set for regression and classification problems, and the simulation results verify the effectiveness of the method.
ISSN:	0952-1976 1873-6769
DOI:	10.1016/j.engappai.2024.108444