Deterministic convergence analysis for regularized long short-term memory and its application to regression and multi-classification problems
Long short-term memory (LSTM) is a recurrent neural network (RNN) framework designed to solve the gradient disappearance and explosion problems of traditional RNNs. In recent years, LSTM has become a state-of-the-art model for solving various machine-learning problems. This paper propose a novel reg...
Uložené v:
| Vydané v: | Engineering applications of artificial intelligence Ročník 133; s. 108444 |
|---|---|
| Hlavní autori: | , , , |
| Médium: | Journal Article |
| Jazyk: | English |
| Vydavateľské údaje: |
Elsevier Ltd
01.07.2024
|
| Predmet: | |
| ISSN: | 0952-1976, 1873-6769 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | Long short-term memory (LSTM) is a recurrent neural network (RNN) framework designed to solve the gradient disappearance and explosion problems of traditional RNNs. In recent years, LSTM has become a state-of-the-art model for solving various machine-learning problems. This paper propose a novel regularized LSTM based on the batch gradient method. Specifically, the L2 regularization is appended to the objective function as a systematic external force, effectively controlling the excessive growth of weights in the network and preventing the overfitting phenomenon. In addition, a rigorous convergence analysis of the proposed method is carried out, i.e., monotonicity, weak convergence, and strong convergence results are obtained. Finally, comparative simulations are conducted on the benchmark data set for regression and classification problems, and the simulation results verify the effectiveness of the method. |
|---|---|
| AbstractList | Long short-term memory (LSTM) is a recurrent neural network (RNN) framework designed to solve the gradient disappearance and explosion problems of traditional RNNs. In recent years, LSTM has become a state-of-the-art model for solving various machine-learning problems. This paper propose a novel regularized LSTM based on the batch gradient method. Specifically, the L2 regularization is appended to the objective function as a systematic external force, effectively controlling the excessive growth of weights in the network and preventing the overfitting phenomenon. In addition, a rigorous convergence analysis of the proposed method is carried out, i.e., monotonicity, weak convergence, and strong convergence results are obtained. Finally, comparative simulations are conducted on the benchmark data set for regression and classification problems, and the simulation results verify the effectiveness of the method. |
| ArticleNumber | 108444 |
| Author | Yu, Dengxiu Cheong, Kang Hao Kang, Qian Wang, Zhen |
| Author_xml | – sequence: 1 givenname: Qian surname: Kang fullname: Kang, Qian email: kangqian0373@126.com organization: School of the Cybersecurity, Northwestern Polytechnical University, Xi’an, 710072, China – sequence: 2 givenname: Dengxiu orcidid: 0000-0003-1803-3946 surname: Yu fullname: Yu, Dengxiu email: yudengxiu@126.com organization: School of Artificial Intelligence, Optics and Electronics (iOPEN), Northwestern Polytechnical University, Xi’an, 710072, China – sequence: 3 givenname: Kang Hao surname: Cheong fullname: Cheong, Kang Hao email: kanghao_cheong@sutd.edu.sg organization: Science, Mathematics and Technology Cluster, Singapore University of Technology and Design, 8 Somapah Road, S487372, Singapore – sequence: 4 givenname: Zhen surname: Wang fullname: Wang, Zhen email: zhenwang0@gmail.com organization: School of Artificial Intelligence, Optics and Electronics (iOPEN), Northwestern Polytechnical University, Xi’an, 710072, China |
| BookMark | eNqFkE1OwzAQRi1UJNrCFZAvkGI7qZNILED8S5XYwNpynEmZyrEr2yCVO3BnEko3bFiNZvS90cybkYnzDgg552zBGZcXmwW4td5uNS4EE8UwrIqiOCJTXpV5JktZT8iU1UuR8bqUJ2QW44YxlleFnJKvW0gQenQYExpqvPuAsAZngGqn7S5ipJ0PNMD63eqAn9BS692axjcfUjaytIfeh92QbymmSIdTLBqd0Dua_EgGiHHsxkT_bhNmxuph1B1i2-AbC308JcedthHOfuucvN7fvdw8Zqvnh6eb61Vmci5SVkvRMak117LgZlnwOhd1Y5aC57rh2rQVZ7oroJI1LyuR85KXDVSNaLq8MQ3L50Tu95rgYwzQqW3AXoed4kyNUtVGHaSqUaraSx3Ayz-gwfTzQwoa7f_41R6H4bkPhKCiwVF2iwFMUq3H_1Z8A3dmnow |
| CitedBy_id | crossref_primary_10_1016_j_ijhydene_2025_151304 crossref_primary_10_1016_j_oceaneng_2025_120676 crossref_primary_10_1007_s11071_025_11792_y |
| Cites_doi | 10.1016/j.neucom.2013.08.005 10.1109/72.279181 10.1109/ACCESS.2022.3228600 10.1007/s11042-020-09198-6 10.1109/TNNLS.2012.2197412 10.1137/S0097539792240406 10.1109/TCYB.2019.2950105 10.1016/j.jhydrol.2023.129229 10.1162/089976600300015763 10.1007/s00521-014-1730-x 10.1162/089976600300015015 10.1016/j.patrec.2022.04.038 10.3390/hydrology9020036 10.1016/j.watres.2022.119100 10.1016/S0925-2312(01)00706-8 10.1088/1361-6420/33/1/015004 10.1109/TITS.2020.3008612 10.1002/int.22590 10.1109/ACCESS.2020.3039539 10.1007/s10589-017-9916-7 10.1016/j.ins.2021.12.039 10.1109/TKDE.2017.2720734 10.1016/j.patcog.2022.108785 10.1111/j.2517-6161.1996.tb02080.x 10.1007/s10489-021-02518-9 10.3390/math10030488 10.1186/s40537-021-00444-8 10.1109/TMM.2020.2978637 10.1016/j.ins.2020.12.014 10.1016/j.neunet.2012.04.013 10.1109/TIP.2013.2262292 10.1016/j.jco.2009.01.002 10.1016/j.asoc.2017.07.059 10.1162/neco.1997.9.8.1735 10.1109/72.883412 10.1109/72.963769 10.1016/j.ins.2021.11.044 10.1016/j.knosys.2022.109526 10.32604/iasc.2020.013918 10.1109/TITS.2011.2119483 10.1016/j.neucom.2012.02.029 |
| ContentType | Journal Article |
| Copyright | 2024 Elsevier Ltd |
| Copyright_xml | – notice: 2024 Elsevier Ltd |
| DBID | AAYXX CITATION |
| DOI | 10.1016/j.engappai.2024.108444 |
| DatabaseName | CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Applied Sciences Computer Science |
| EISSN | 1873-6769 |
| ExternalDocumentID | 10_1016_j_engappai_2024_108444 S095219762400602X |
| GrantInformation_xml | – fundername: Natural Science Foundation of Shaanxi Province grantid: 2024JC-YBQN-0663 funderid: http://dx.doi.org/10.13039/501100007128 – fundername: Tencent Foundation and XPLORER PRIZE – fundername: National Natural Science Foundation of China grantid: U22B2036; 11931015; 62373302; 62333009 funderid: http://dx.doi.org/10.13039/501100001809 – fundername: Fok Ying-Tong Education Foundationm China grantid: 171105 – fundername: National Science Fund for Distinguished Young Scholarship of China grantid: 62025602 – fundername: Technology Innovation Leading Program of Shaanxi grantid: 2023GXLH-086 |
| GroupedDBID | --K --M .DC .~1 0R~ 1B1 1~. 1~5 29G 4.4 457 4G. 5GY 5VS 7-5 71M 8P~ 9JN AABNK AACTN AAEDT AAEDW AAIAV AAIKJ AAKOC AALRI AAOAW AAQFI AAQXK AAXUO AAYFN ABBOA ABMAC ABXDB ACDAQ ACGFS ACNNM ACRLP ACZNC ADBBV ADEZE ADJOM ADMUD ADTZH AEBSH AECPX AEKER AENEX AFKWA AFTJW AGHFR AGUBO AGYEJ AHHHB AHJVU AHZHX AIALX AIEXJ AIKHN AITUG AJOXV AKRWK ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AOUOD ASPBG AVWKF AXJTR AZFZN BJAXD BKOJK BLXMC CS3 DU5 EBS EFJIC EJD EO8 EO9 EP2 EP3 F5P FDB FEDTE FGOYB FIRID FNPLU FYGXN G-2 G-Q GBLVA GBOLZ HLZ HVGLF HZ~ IHE J1W JJJVA KOM LG9 LY7 M41 MO0 N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. Q38 R2- RIG ROL RPZ SBC SDF SDG SDP SES SET SEW SPC SPCBC SST SSV SSZ T5K TN5 UHS WUQ ZMT ~G- 9DU AATTM AAXKI AAYWO AAYXX ABJNI ABWVN ACLOT ACRPL ACVFH ADCNI ADNMO AEIPS AEUPX AFJKZ AFPUW AGQPQ AIGII AIIUN AKBMS AKYEP ANKPU APXCP CITATION EFKBS EFLBG ~HD |
| ID | FETCH-LOGICAL-c312t-962f06aa1a641c5419329bc5213ab1acd810af4e869178231717be8b2bf3bcb03 |
| ISICitedReferencesCount | 7 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001236651400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0952-1976 |
| IngestDate | Tue Nov 18 21:01:22 EST 2025 Sat Nov 29 03:41:18 EST 2025 Tue Jun 18 08:50:47 EDT 2024 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Long short-term memory Batch gradient algorithm Regularization Convergence |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c312t-962f06aa1a641c5419329bc5213ab1acd810af4e869178231717be8b2bf3bcb03 |
| ORCID | 0000-0003-1803-3946 |
| ParticipantIDs | crossref_primary_10_1016_j_engappai_2024_108444 crossref_citationtrail_10_1016_j_engappai_2024_108444 elsevier_sciencedirect_doi_10_1016_j_engappai_2024_108444 |
| PublicationCentury | 2000 |
| PublicationDate | July 2024 2024-07-00 |
| PublicationDateYYYYMMDD | 2024-07-01 |
| PublicationDate_xml | – month: 07 year: 2024 text: July 2024 |
| PublicationDecade | 2020 |
| PublicationTitle | Engineering applications of artificial intelligence |
| PublicationYear | 2024 |
| Publisher | Elsevier Ltd |
| Publisher_xml | – name: Elsevier Ltd |
| References | Wollmer, Blaschke, Schindl (b37) 2011; 12 Saito, Nakano (b27) 2000; 12 Wang, Hao, Zhang (b33) 2022; 588 Husken, Stagge (b15) 2003; 50 Donnelly, Abolfathi, Pearson (b7) 2022; 225 Chen, Wang, Xu (b4) 2022; 52 Gers, Schmidhuber, Cummins (b10) 2000; 12 Ludwig, Nunes, Araujo (b22) 2014; 124 Bengio, Simard, Frasconi (b2) 1994; 5 Wang, Wu, Zurada (b36) 2012; 33 Hochreiter, Schmidhuber (b14) 1997; 9 Lee, Kim, Lee (b19) 2020; 23 Zhang, Wu, Yao (b45) 2012; 89 Fan, Kang, Zurada (b8) 2022; 585 Xie, Li, Diao, An, Xu (b39) 2019; 11 Jian, Xiang, Le (b16) 2022; 2022 Haydari, Yilmaz (b13) 2020; 23 Li, Fang, Zha, Gao, Zheng (b20) 2022; 129 Gers, Schmidhuber (b9) 2001; 12 Natarajan (b25) 1995; 24 Thakkar, Lohiya, Zhang (b30) 2021; 36 Cheng, Chen, Xiao (b5) 2022 Khosravi, Rezaie, Cooper (b18) 2023; 618 Maragheh, Gharehchopogh, Majidzadeh (b24) 2022; 10 Xie, Zhang, Wang, Wang, Pal (b40) 2020; 50 Zhang, Ye, Zhang (b46) 2017; 68 Zhang, Tang, Liu (b44) 2015; 26 Yang, Zhou, Balasubramanian (b43) 2013; 22 Guptha, Balamurugan, Megharaj (b12) 2022; 159 Liang, Wang (b21) 2000; 11 Chen, Hofmann, Zou (b3) 2017; 33 Noori, Ghiasi, Salehi (b26) 2022; 9 Wang, Wen, Ye (b34) 2017; 61 Xu, Chang, Xu (b41) 2012; 23 Kang, Fan, Zurada (b17) 2021; 553 Vijayaprabakaran, Sathiyamurthy (b32) 2020; 34 Wang, Wu, Zhang (b35) 2022 Stuner, Chatelain, Paquet (b29) 2020; 79 Guo (b11) 2020; 26 Shi, Chen, Chen, Lee (b28) 2022; 10 Yang, Yu, Ma (b42) 2022; 253 Luo, Liu, Yin, Li, Wu (b23) 2017; 29 De Mol, De Vito, Rosasco (b6) 2009; 25 Alzubaidi, Zhang, Humaidi (b1) 2021; 8 Xiao, Chang, Zhang (b38) 2020; 8 Tibshirani (b31) 1996; 58 Shi (10.1016/j.engappai.2024.108444_b28) 2022; 10 Wollmer (10.1016/j.engappai.2024.108444_b37) 2011; 12 Maragheh (10.1016/j.engappai.2024.108444_b24) 2022; 10 Bengio (10.1016/j.engappai.2024.108444_b2) 1994; 5 Donnelly (10.1016/j.engappai.2024.108444_b7) 2022; 225 Alzubaidi (10.1016/j.engappai.2024.108444_b1) 2021; 8 Saito (10.1016/j.engappai.2024.108444_b27) 2000; 12 Khosravi (10.1016/j.engappai.2024.108444_b18) 2023; 618 Kang (10.1016/j.engappai.2024.108444_b17) 2021; 553 Lee (10.1016/j.engappai.2024.108444_b19) 2020; 23 Vijayaprabakaran (10.1016/j.engappai.2024.108444_b32) 2020; 34 Zhang (10.1016/j.engappai.2024.108444_b44) 2015; 26 Zhang (10.1016/j.engappai.2024.108444_b45) 2012; 89 Husken (10.1016/j.engappai.2024.108444_b15) 2003; 50 Zhang (10.1016/j.engappai.2024.108444_b46) 2017; 68 Yang (10.1016/j.engappai.2024.108444_b42) 2022; 253 Gers (10.1016/j.engappai.2024.108444_b10) 2000; 12 Yang (10.1016/j.engappai.2024.108444_b43) 2013; 22 Chen (10.1016/j.engappai.2024.108444_b3) 2017; 33 Cheng (10.1016/j.engappai.2024.108444_b5) 2022 Luo (10.1016/j.engappai.2024.108444_b23) 2017; 29 Wang (10.1016/j.engappai.2024.108444_b34) 2017; 61 Fan (10.1016/j.engappai.2024.108444_b8) 2022; 585 Thakkar (10.1016/j.engappai.2024.108444_b30) 2021; 36 Tibshirani (10.1016/j.engappai.2024.108444_b31) 1996; 58 Xu (10.1016/j.engappai.2024.108444_b41) 2012; 23 Wang (10.1016/j.engappai.2024.108444_b36) 2012; 33 Guo (10.1016/j.engappai.2024.108444_b11) 2020; 26 De Mol (10.1016/j.engappai.2024.108444_b6) 2009; 25 Guptha (10.1016/j.engappai.2024.108444_b12) 2022; 159 Ludwig (10.1016/j.engappai.2024.108444_b22) 2014; 124 Gers (10.1016/j.engappai.2024.108444_b9) 2001; 12 Haydari (10.1016/j.engappai.2024.108444_b13) 2020; 23 Xie (10.1016/j.engappai.2024.108444_b40) 2020; 50 Li (10.1016/j.engappai.2024.108444_b20) 2022; 129 Hochreiter (10.1016/j.engappai.2024.108444_b14) 1997; 9 Chen (10.1016/j.engappai.2024.108444_b4) 2022; 52 Stuner (10.1016/j.engappai.2024.108444_b29) 2020; 79 Xiao (10.1016/j.engappai.2024.108444_b38) 2020; 8 Wang (10.1016/j.engappai.2024.108444_b33) 2022; 588 Liang (10.1016/j.engappai.2024.108444_b21) 2000; 11 Xie (10.1016/j.engappai.2024.108444_b39) 2019; 11 Noori (10.1016/j.engappai.2024.108444_b26) 2022; 9 Natarajan (10.1016/j.engappai.2024.108444_b25) 1995; 24 Jian (10.1016/j.engappai.2024.108444_b16) 2022; 2022 Wang (10.1016/j.engappai.2024.108444_b35) 2022 |
| References_xml | – volume: 50 start-page: 223 year: 2003 end-page: 235 ident: b15 article-title: Recurrent neural networks for time series classification publication-title: Neurocomputing – volume: 618 year: 2023 ident: b18 article-title: Soil water erosion susceptibility assessment using deep learning algorithms publication-title: J. Hydrol. – volume: 29 start-page: 2125 year: 2017 end-page: 2139 ident: b23 article-title: Deep learning of graphs with ngram convolutional neural networks publication-title: IEEE Trans. Knowl. Data Eng. – volume: 10 start-page: 488 year: 2022 ident: b24 article-title: A new hybrid based on long short-term memory network with spotted hyena optimization algorithm for multi-label text classification publication-title: Mathematics – volume: 588 start-page: 106 year: 2022 end-page: 123 ident: b33 article-title: Convergence and robustness of bounded recurrent neural networks for solving dynamic Lyapunov equations publication-title: Inform. Sci. – volume: 12 start-page: 2451 year: 2000 end-page: 2471 ident: b10 article-title: Learning to forget: Continual prediction with LSTM publication-title: Neural Comput. – volume: 52 start-page: 7513 year: 2022 end-page: 7528 ident: b4 article-title: GC-LSTM: Graph convolution embedded LSTM for dynamic network link prediction publication-title: Appl. Intell. – volume: 9 start-page: 1735 year: 1997 end-page: 1780 ident: b14 article-title: Long short-term memory publication-title: Neural Comput. – volume: 22 start-page: 3234 year: 2013 end-page: 3246 ident: b43 article-title: Fast publication-title: IEEE Trans. Image Process. – volume: 8 start-page: 1 year: 2021 end-page: 74 ident: b1 article-title: Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions publication-title: J. Big Data – volume: 12 start-page: 1333 year: 2001 end-page: 1340 ident: b9 article-title: LSTM recurrent networks learn simple context-free and context-sensitive languages publication-title: IEEE Trans. Neural Netw. Learn. Syst. – volume: 26 start-page: 383 year: 2015 end-page: 390 ident: b44 article-title: Batch gradient training method with smoothing publication-title: Neural Comput. Appl. – volume: 129 year: 2022 ident: b20 article-title: HAM: Hybrid attention module in deep convolutional neural networks for image classification publication-title: Pattern Recognit. – volume: 25 start-page: 201 year: 2009 end-page: 230 ident: b6 article-title: Elastic-net regularization in learning theory publication-title: J. Complexity – volume: 159 start-page: 16 year: 2022 end-page: 22 ident: b12 article-title: Cross lingual handwritten character recognition using long short term memory network with aid of elephant herding optimization algorithm publication-title: Pattern Recognit. Lett. – volume: 8 year: 2020 ident: b38 article-title: Multi-information spatial–temporal LSTM fusion continuous sign language neural machine translation publication-title: IEEE Access – volume: 68 start-page: 437 year: 2017 end-page: 454 ident: b46 article-title: A generalized elastic net regularization with smoothed publication-title: Comput. Optim. Appl. – volume: 89 start-page: 141 year: 2012 end-page: 146 ident: b45 article-title: Boundedness and convergence of batch backpropagation algorithm with penalty for feedforward neural networks publication-title: Neurocomputing – volume: 11 start-page: 1 year: 2019 end-page: 4 ident: b39 article-title: Regularization based fine-grained neural network pruning method publication-title: Proc. Int. Conf. Electron. Comput. Artif. Intell. – volume: 585 start-page: 70 year: 2022 end-page: 88 ident: b8 article-title: Convergence analysis for Sigma-Pi-Sigma neural network based on some relaxed conditions publication-title: Inform. Sci. – volume: 12 start-page: 709 year: 2000 end-page: 729 ident: b27 article-title: Second-order learning algorithm with squared penalty term publication-title: Neural Comput. – volume: 61 start-page: 354 year: 2017 end-page: 363 ident: b34 article-title: Convergence analysis of BP neural networks via sparse response regularization publication-title: Appl. Soft Comput. – volume: 26 start-page: 421 year: 2020 end-page: 427 ident: b11 article-title: Extreme learning machine with elastic net regularization publication-title: Intell. Autom. Soft Comput. – volume: 124 start-page: 33 year: 2014 end-page: 42 ident: b22 article-title: Eigenvalue decay: A new method for neural network regularization publication-title: Neurocomputing – volume: 58 start-page: 267 year: 1996 end-page: 288 ident: b31 article-title: Regression shrinkage and selection via the lasso publication-title: J. R. Stat. Soc. Ser. B Methodol. – volume: 12 start-page: 574 year: 2011 end-page: 582 ident: b37 article-title: Online driver distraction detection using long short-term memory publication-title: IEEE Trans. Intell. Transp. Syst. – volume: 24 start-page: 227 year: 1995 end-page: 234 ident: b25 article-title: Sparse approximate solutions to linear systems publication-title: SIAM J. Comput. – volume: 33 year: 2017 ident: b3 article-title: Elastic-net regularization versus publication-title: Inverse Probl. – volume: 2022 year: 2022 ident: b16 article-title: LSTM-based attentional embedding for english machine translation publication-title: Sci. Program. – volume: 79 start-page: 34407 year: 2020 end-page: 34427 ident: b29 article-title: Handwriting recognition using cohort of LSTM and lexicon verification with extremely large lexicon publication-title: Multimed. Tools Appl. – start-page: 1 year: 2022 end-page: 22 ident: b5 article-title: A dual-stage attention-based Bi-LSTM network for multivariate time series prediction publication-title: J. Supercomput. – year: 2022 ident: b35 article-title: Predrnn: A recurrent neural network for spatiotemporal predictive learning publication-title: IEEE Trans. Pattern Anal. Mach. Intell. – volume: 9 year: 2022 ident: b26 article-title: An efficient data driven-based model for prediction of the total sediment load in rivers publication-title: Hydrology – volume: 225 year: 2022 ident: b7 article-title: Gaussian process emulation of spatio-temporal outputs of a 2D inland flood model publication-title: Water Res. – volume: 11 start-page: 1251 year: 2000 end-page: 1262 ident: b21 article-title: A recurrent neural network for nonlinear optimization with a continuously differentiable objective function and bound constraints publication-title: IEEE Trans. Neural Netw. Learn. Syst. – volume: 5 start-page: 157 year: 1994 end-page: 166 ident: b2 article-title: Learning long-term dependencies with gradient descent is difficult publication-title: IEEE Trans. Neural Netw. Learn. Syst. – volume: 10 year: 2022 ident: b28 article-title: CNO-LSTM: A chaotic neural oscillatory long short-term memory model for text classification publication-title: IEEE Access – volume: 553 start-page: 66 year: 2021 end-page: 82 ident: b17 article-title: Deterministic convergence analysis via smoothing group Lasso regularization and adaptive momentum for Sigma-Pi-Sigma neural network publication-title: Inform. Sci. – volume: 253 year: 2022 ident: b42 article-title: Deep representation-based transfer learning for deep neural networks publication-title: Knowl.-Based Syst. – volume: 33 start-page: 127 year: 2012 end-page: 135 ident: b36 article-title: Computational properties and convergence analysis of BPNN for cyclic and almost cyclic learning with penalty publication-title: Neural Netw. – volume: 23 start-page: 11 year: 2020 end-page: 32 ident: b13 article-title: Deep reinforcement learning for intelligent transportation systems: A survey publication-title: IEEE Trans. Intell. Transp. Syst. – volume: 50 start-page: 1333 year: 2020 end-page: 1346 ident: b40 article-title: Learning optimized structure of neural networks by hidden node pruning with publication-title: IEEE Trans Cybern. – volume: 23 start-page: 1013 year: 2012 end-page: 1027 ident: b41 article-title: Regularization: A thresholding representation theory and a fast solver publication-title: IEEE Trans. Neural Netw. Learn. Syst. – volume: 34 start-page: 2637 year: 2020 end-page: 2650 ident: b32 article-title: Towards activation function search for long short-term model network: A differential evolution based approach publication-title: J. King Saud Univ.-Comput. Inf. Sci. – volume: 23 start-page: 415 year: 2020 end-page: 428 ident: b19 article-title: 3-d human behavior understanding using generalized ts-lstm networks publication-title: IEEE Trans. Multimed. – volume: 36 year: 2021 ident: b30 article-title: Analyzing fusion of regularization techniques in the deep learning-based intrusion detection system publication-title: Int. J. Intell. Syst. – volume: 124 start-page: 33 year: 2014 ident: 10.1016/j.engappai.2024.108444_b22 article-title: Eigenvalue decay: A new method for neural network regularization publication-title: Neurocomputing doi: 10.1016/j.neucom.2013.08.005 – volume: 2022 year: 2022 ident: 10.1016/j.engappai.2024.108444_b16 article-title: LSTM-based attentional embedding for english machine translation publication-title: Sci. Program. – volume: 5 start-page: 157 issue: 2 year: 1994 ident: 10.1016/j.engappai.2024.108444_b2 article-title: Learning long-term dependencies with gradient descent is difficult publication-title: IEEE Trans. Neural Netw. Learn. Syst. doi: 10.1109/72.279181 – volume: 10 year: 2022 ident: 10.1016/j.engappai.2024.108444_b28 article-title: CNO-LSTM: A chaotic neural oscillatory long short-term memory model for text classification publication-title: IEEE Access doi: 10.1109/ACCESS.2022.3228600 – volume: 11 start-page: 1 year: 2019 ident: 10.1016/j.engappai.2024.108444_b39 article-title: L0 Regularization based fine-grained neural network pruning method publication-title: Proc. Int. Conf. Electron. Comput. Artif. Intell. – volume: 79 start-page: 34407 issue: 45 year: 2020 ident: 10.1016/j.engappai.2024.108444_b29 article-title: Handwriting recognition using cohort of LSTM and lexicon verification with extremely large lexicon publication-title: Multimed. Tools Appl. doi: 10.1007/s11042-020-09198-6 – volume: 23 start-page: 1013 issue: 7 year: 2012 ident: 10.1016/j.engappai.2024.108444_b41 article-title: L1/2 Regularization: A thresholding representation theory and a fast solver publication-title: IEEE Trans. Neural Netw. Learn. Syst. doi: 10.1109/TNNLS.2012.2197412 – volume: 24 start-page: 227 issue: 2 year: 1995 ident: 10.1016/j.engappai.2024.108444_b25 article-title: Sparse approximate solutions to linear systems publication-title: SIAM J. Comput. doi: 10.1137/S0097539792240406 – volume: 50 start-page: 1333 year: 2020 ident: 10.1016/j.engappai.2024.108444_b40 article-title: Learning optimized structure of neural networks by hidden node pruning with L1 regularization publication-title: IEEE Trans Cybern. doi: 10.1109/TCYB.2019.2950105 – volume: 618 year: 2023 ident: 10.1016/j.engappai.2024.108444_b18 article-title: Soil water erosion susceptibility assessment using deep learning algorithms publication-title: J. Hydrol. doi: 10.1016/j.jhydrol.2023.129229 – volume: 12 start-page: 709 issue: 3 year: 2000 ident: 10.1016/j.engappai.2024.108444_b27 article-title: Second-order learning algorithm with squared penalty term publication-title: Neural Comput. doi: 10.1162/089976600300015763 – volume: 26 start-page: 383 issue: 2 year: 2015 ident: 10.1016/j.engappai.2024.108444_b44 article-title: Batch gradient training method with smoothing L0 regularization for feedforward neural networks publication-title: Neural Comput. Appl. doi: 10.1007/s00521-014-1730-x – volume: 12 start-page: 2451 issue: 10 year: 2000 ident: 10.1016/j.engappai.2024.108444_b10 article-title: Learning to forget: Continual prediction with LSTM publication-title: Neural Comput. doi: 10.1162/089976600300015015 – volume: 159 start-page: 16 year: 2022 ident: 10.1016/j.engappai.2024.108444_b12 article-title: Cross lingual handwritten character recognition using long short term memory network with aid of elephant herding optimization algorithm publication-title: Pattern Recognit. Lett. doi: 10.1016/j.patrec.2022.04.038 – volume: 9 year: 2022 ident: 10.1016/j.engappai.2024.108444_b26 article-title: An efficient data driven-based model for prediction of the total sediment load in rivers publication-title: Hydrology doi: 10.3390/hydrology9020036 – volume: 225 year: 2022 ident: 10.1016/j.engappai.2024.108444_b7 article-title: Gaussian process emulation of spatio-temporal outputs of a 2D inland flood model publication-title: Water Res. doi: 10.1016/j.watres.2022.119100 – volume: 50 start-page: 223 year: 2003 ident: 10.1016/j.engappai.2024.108444_b15 article-title: Recurrent neural networks for time series classification publication-title: Neurocomputing doi: 10.1016/S0925-2312(01)00706-8 – volume: 33 year: 2017 ident: 10.1016/j.engappai.2024.108444_b3 article-title: Elastic-net regularization versus l1-regularization for linear inverse problems with quasi-sparse solutions publication-title: Inverse Probl. doi: 10.1088/1361-6420/33/1/015004 – volume: 23 start-page: 11 issue: 1 year: 2020 ident: 10.1016/j.engappai.2024.108444_b13 article-title: Deep reinforcement learning for intelligent transportation systems: A survey publication-title: IEEE Trans. Intell. Transp. Syst. doi: 10.1109/TITS.2020.3008612 – volume: 36 year: 2021 ident: 10.1016/j.engappai.2024.108444_b30 article-title: Analyzing fusion of regularization techniques in the deep learning-based intrusion detection system publication-title: Int. J. Intell. Syst. doi: 10.1002/int.22590 – volume: 8 year: 2020 ident: 10.1016/j.engappai.2024.108444_b38 article-title: Multi-information spatial–temporal LSTM fusion continuous sign language neural machine translation publication-title: IEEE Access doi: 10.1109/ACCESS.2020.3039539 – volume: 68 start-page: 437 year: 2017 ident: 10.1016/j.engappai.2024.108444_b46 article-title: A generalized elastic net regularization with smoothed lq penalty for sparse vector recovery publication-title: Comput. Optim. Appl. doi: 10.1007/s10589-017-9916-7 – volume: 588 start-page: 106 year: 2022 ident: 10.1016/j.engappai.2024.108444_b33 article-title: Convergence and robustness of bounded recurrent neural networks for solving dynamic Lyapunov equations publication-title: Inform. Sci. doi: 10.1016/j.ins.2021.12.039 – volume: 29 start-page: 2125 issue: 10 year: 2017 ident: 10.1016/j.engappai.2024.108444_b23 article-title: Deep learning of graphs with ngram convolutional neural networks publication-title: IEEE Trans. Knowl. Data Eng. doi: 10.1109/TKDE.2017.2720734 – volume: 129 year: 2022 ident: 10.1016/j.engappai.2024.108444_b20 article-title: HAM: Hybrid attention module in deep convolutional neural networks for image classification publication-title: Pattern Recognit. doi: 10.1016/j.patcog.2022.108785 – volume: 58 start-page: 267 year: 1996 ident: 10.1016/j.engappai.2024.108444_b31 article-title: Regression shrinkage and selection via the lasso publication-title: J. R. Stat. Soc. Ser. B Methodol. doi: 10.1111/j.2517-6161.1996.tb02080.x – volume: 52 start-page: 7513 issue: 7 year: 2022 ident: 10.1016/j.engappai.2024.108444_b4 article-title: GC-LSTM: Graph convolution embedded LSTM for dynamic network link prediction publication-title: Appl. Intell. doi: 10.1007/s10489-021-02518-9 – volume: 34 start-page: 2637 issue: 6 year: 2020 ident: 10.1016/j.engappai.2024.108444_b32 article-title: Towards activation function search for long short-term model network: A differential evolution based approach publication-title: J. King Saud Univ.-Comput. Inf. Sci. – volume: 10 start-page: 488 issue: 3 year: 2022 ident: 10.1016/j.engappai.2024.108444_b24 article-title: A new hybrid based on long short-term memory network with spotted hyena optimization algorithm for multi-label text classification publication-title: Mathematics doi: 10.3390/math10030488 – start-page: 1 year: 2022 ident: 10.1016/j.engappai.2024.108444_b5 article-title: A dual-stage attention-based Bi-LSTM network for multivariate time series prediction publication-title: J. Supercomput. – volume: 8 start-page: 1 issue: 1 year: 2021 ident: 10.1016/j.engappai.2024.108444_b1 article-title: Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions publication-title: J. Big Data doi: 10.1186/s40537-021-00444-8 – volume: 23 start-page: 415 year: 2020 ident: 10.1016/j.engappai.2024.108444_b19 article-title: 3-d human behavior understanding using generalized ts-lstm networks publication-title: IEEE Trans. Multimed. doi: 10.1109/TMM.2020.2978637 – volume: 553 start-page: 66 year: 2021 ident: 10.1016/j.engappai.2024.108444_b17 article-title: Deterministic convergence analysis via smoothing group Lasso regularization and adaptive momentum for Sigma-Pi-Sigma neural network publication-title: Inform. Sci. doi: 10.1016/j.ins.2020.12.014 – volume: 33 start-page: 127 year: 2012 ident: 10.1016/j.engappai.2024.108444_b36 article-title: Computational properties and convergence analysis of BPNN for cyclic and almost cyclic learning with penalty publication-title: Neural Netw. doi: 10.1016/j.neunet.2012.04.013 – volume: 22 start-page: 3234 issue: 8 year: 2013 ident: 10.1016/j.engappai.2024.108444_b43 article-title: Fast ℓ1-minimization algorithms for robust face recognition publication-title: IEEE Trans. Image Process. doi: 10.1109/TIP.2013.2262292 – volume: 25 start-page: 201 year: 2009 ident: 10.1016/j.engappai.2024.108444_b6 article-title: Elastic-net regularization in learning theory publication-title: J. Complexity doi: 10.1016/j.jco.2009.01.002 – volume: 61 start-page: 354 year: 2017 ident: 10.1016/j.engappai.2024.108444_b34 article-title: Convergence analysis of BP neural networks via sparse response regularization publication-title: Appl. Soft Comput. doi: 10.1016/j.asoc.2017.07.059 – volume: 9 start-page: 1735 issue: 8 year: 1997 ident: 10.1016/j.engappai.2024.108444_b14 article-title: Long short-term memory publication-title: Neural Comput. doi: 10.1162/neco.1997.9.8.1735 – year: 2022 ident: 10.1016/j.engappai.2024.108444_b35 article-title: Predrnn: A recurrent neural network for spatiotemporal predictive learning publication-title: IEEE Trans. Pattern Anal. Mach. Intell. – volume: 11 start-page: 1251 issue: 6 year: 2000 ident: 10.1016/j.engappai.2024.108444_b21 article-title: A recurrent neural network for nonlinear optimization with a continuously differentiable objective function and bound constraints publication-title: IEEE Trans. Neural Netw. Learn. Syst. doi: 10.1109/72.883412 – volume: 12 start-page: 1333 issue: 6 year: 2001 ident: 10.1016/j.engappai.2024.108444_b9 article-title: LSTM recurrent networks learn simple context-free and context-sensitive languages publication-title: IEEE Trans. Neural Netw. Learn. Syst. doi: 10.1109/72.963769 – volume: 585 start-page: 70 year: 2022 ident: 10.1016/j.engappai.2024.108444_b8 article-title: Convergence analysis for Sigma-Pi-Sigma neural network based on some relaxed conditions publication-title: Inform. Sci. doi: 10.1016/j.ins.2021.11.044 – volume: 253 year: 2022 ident: 10.1016/j.engappai.2024.108444_b42 article-title: Deep representation-based transfer learning for deep neural networks publication-title: Knowl.-Based Syst. doi: 10.1016/j.knosys.2022.109526 – volume: 26 start-page: 421 year: 2020 ident: 10.1016/j.engappai.2024.108444_b11 article-title: Extreme learning machine with elastic net regularization publication-title: Intell. Autom. Soft Comput. doi: 10.32604/iasc.2020.013918 – volume: 12 start-page: 574 issue: 2 year: 2011 ident: 10.1016/j.engappai.2024.108444_b37 article-title: Online driver distraction detection using long short-term memory publication-title: IEEE Trans. Intell. Transp. Syst. doi: 10.1109/TITS.2011.2119483 – volume: 89 start-page: 141 year: 2012 ident: 10.1016/j.engappai.2024.108444_b45 article-title: Boundedness and convergence of batch backpropagation algorithm with penalty for feedforward neural networks publication-title: Neurocomputing doi: 10.1016/j.neucom.2012.02.029 |
| SSID | ssj0003846 |
| Score | 2.4393427 |
| Snippet | Long short-term memory (LSTM) is a recurrent neural network (RNN) framework designed to solve the gradient disappearance and explosion problems of traditional... |
| SourceID | crossref elsevier |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 108444 |
| SubjectTerms | Batch gradient algorithm Convergence Long short-term memory Regularization |
| Title | Deterministic convergence analysis for regularized long short-term memory and its application to regression and multi-classification problems |
| URI | https://dx.doi.org/10.1016/j.engappai.2024.108444 |
| Volume | 133 |
| WOSCitedRecordID | wos001236651400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 customDbUrl: eissn: 1873-6769 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0003846 issn: 0952-1976 databaseCode: AIEXJ dateStart: 19950201 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV3NbtQwELaWlgMXKH9qgSIfuFUpifNj-1hBUeFQgSjSwiWynSxNtWSr7G618A48ES_HTGwnWVipIMQlWlmeOMl8a4_H38wQ8iwTRRIKbYKQa9ygmCyQHHatnClRlBxs8ky3xSb46akYj-Xb0eiHj4W5mvK6FquVvPyvqoY2UDaGzv6FurubQgP8BqXDFdQO1z9S_EtHcGkzMFtWeWMzbiqfgASphU1bhL6pvoHFOcWCQ_NzsMQDlD34gvTbr925wuCQG01VkLTkWctjbimJgUErHGlHtpurUzNfc_z3qQ-Ht2y5JPgeLpdFNUgS2i0Hzqv9bgDmj0s7X9afV9WyZymUjmOMIgcnatafGNj2T-cu-M25OljS0WKd_83H4PSEJ-vIZEEkuUuobadxweMAybtr87zNuPHbmmHdFxeH8Lzw8qo6xKGRe5nYzJS_5ON-jwPieMi-zUI2vkG2GU8lTKnbR6-Px286QyAWNk7MP-AgQH3zaJtto4G9c7ZDbruNCj2yALtLRmV9j9xxmxbqloQ5NPm6IL7tPvm-BkE6gCD1EKQAQTqAIEUI0h6C1EIQ-hcUIEgHeKGLGe0h2PbYBEHqIfiAfHh1fPbiJHBlPwITR2wRyIxNwkypSGVJZNIEtxhSG_josdKRMoWIQjVJSpHJiOMpNo-4LoVmehJro8P4IdmqZ3W5S2jMosJwncWapQkvQpXJVBQmnBSyQLtsj6T-g-fG5cTH0izT3JMfL3KvqBwVlVtF7ZHnndylzQpzrYT0-sydbWtt1hxgeI3so3-QfUxu9f-kJ2Rr0SzLfXLTXC2qefPUIfYn7_jWdA |
| linkProvider | Elsevier |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Deterministic+convergence+analysis+for+regularized+long+short-term+memory+and+its+application+to+regression+and+multi-classification+problems&rft.jtitle=Engineering+applications+of+artificial+intelligence&rft.au=Kang%2C+Qian&rft.au=Yu%2C+Dengxiu&rft.au=Cheong%2C+Kang+Hao&rft.au=Wang%2C+Zhen&rft.date=2024-07-01&rft.pub=Elsevier+Ltd&rft.issn=0952-1976&rft.eissn=1873-6769&rft.volume=133&rft_id=info:doi/10.1016%2Fj.engappai.2024.108444&rft.externalDocID=S095219762400602X |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0952-1976&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0952-1976&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0952-1976&client=summon |