Recurrent neural network from adder’s perspective: Carry-lookahead RNN
The recurrent network architecture is a widely used model in sequence modeling, but its serial dependency hinders the computation parallelization, which makes the operation inefficient. The same problem was encountered in serial adder at the early stage of digital electronics. In this paper, we disc...
Uložené v:
| Vydané v: | Neural networks Ročník 144; s. 297 - 306 |
|---|---|
| Hlavní autori: | , , , , |
| Médium: | Journal Article |
| Jazyk: | English |
| Vydavateľské údaje: |
Elsevier Ltd
01.12.2021
|
| Predmet: | |
| ISSN: | 0893-6080, 1879-2782, 1879-2782 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | The recurrent network architecture is a widely used model in sequence modeling, but its serial dependency hinders the computation parallelization, which makes the operation inefficient. The same problem was encountered in serial adder at the early stage of digital electronics. In this paper, we discuss the similarities between recurrent neural network (RNN) and serial adder. Inspired by carry-lookahead adder, we introduce carry-lookahead module to RNN, which makes it possible for RNN to run in parallel. Then, we design the method of parallel RNN computation, and finally Carry-lookahead RNN (CL-RNN) is proposed. CL-RNN takes advantages in parallelism and flexible receptive field. Through a comprehensive set of tests, we verify that CL-RNN can perform better than existing typical RNNs in sequence modeling tasks which are specially designed for RNNs. Code and models are available at: https://github.com/WinnieJiangHW/Carry-lookahead_RNN. |
|---|---|
| AbstractList | The recurrent network architecture is a widely used model in sequence modeling, but its serial dependency hinders the computation parallelization, which makes the operation inefficient. The same problem was encountered in serial adder at the early stage of digital electronics. In this paper, we discuss the similarities between recurrent neural network (RNN) and serial adder. Inspired by carry-lookahead adder, we introduce carry-lookahead module to RNN, which makes it possible for RNN to run in parallel. Then, we design the method of parallel RNN computation, and finally Carry-lookahead RNN (CL-RNN) is proposed. CL-RNN takes advantages in parallelism and flexible receptive field. Through a comprehensive set of tests, we verify that CL-RNN can perform better than existing typical RNNs in sequence modeling tasks which are specially designed for RNNs. Code and models are available at: https://github.com/WinnieJiangHW/Carry-lookahead_RNN.The recurrent network architecture is a widely used model in sequence modeling, but its serial dependency hinders the computation parallelization, which makes the operation inefficient. The same problem was encountered in serial adder at the early stage of digital electronics. In this paper, we discuss the similarities between recurrent neural network (RNN) and serial adder. Inspired by carry-lookahead adder, we introduce carry-lookahead module to RNN, which makes it possible for RNN to run in parallel. Then, we design the method of parallel RNN computation, and finally Carry-lookahead RNN (CL-RNN) is proposed. CL-RNN takes advantages in parallelism and flexible receptive field. Through a comprehensive set of tests, we verify that CL-RNN can perform better than existing typical RNNs in sequence modeling tasks which are specially designed for RNNs. Code and models are available at: https://github.com/WinnieJiangHW/Carry-lookahead_RNN. The recurrent network architecture is a widely used model in sequence modeling, but its serial dependency hinders the computation parallelization, which makes the operation inefficient. The same problem was encountered in serial adder at the early stage of digital electronics. In this paper, we discuss the similarities between recurrent neural network (RNN) and serial adder. Inspired by carry-lookahead adder, we introduce carry-lookahead module to RNN, which makes it possible for RNN to run in parallel. Then, we design the method of parallel RNN computation, and finally Carry-lookahead RNN (CL-RNN) is proposed. CL-RNN takes advantages in parallelism and flexible receptive field. Through a comprehensive set of tests, we verify that CL-RNN can perform better than existing typical RNNs in sequence modeling tasks which are specially designed for RNNs. Code and models are available at: https://github.com/WinnieJiangHW/Carry-lookahead_RNN. |
| Author | Shao, Yanli Qin, Feiwei Peng, Yong Jiang, Haowei Cao, Jin |
| Author_xml | – sequence: 1 givenname: Haowei orcidid: 0000-0002-5619-5252 surname: Jiang fullname: Jiang, Haowei email: dsfdsf@hdu.edu.cn organization: School of Computer Science and Technology, Hangzhou Dianzi University, China – sequence: 2 givenname: Feiwei surname: Qin fullname: Qin, Feiwei email: qinfeiwei@hdu.edu.cn organization: School of Computer Science and Technology, Hangzhou Dianzi University, China – sequence: 3 givenname: Jin orcidid: 0000-0002-9880-0342 surname: Cao fullname: Cao, Jin email: jcao21@jhu.edu organization: Whiting School of Engineering, Johns Hopkins University, USA – sequence: 4 givenname: Yong orcidid: 0000-0003-1208-972X surname: Peng fullname: Peng, Yong email: yongpeng@hdu.edu.cn organization: School of Computer Science and Technology, Hangzhou Dianzi University, China – sequence: 5 givenname: Yanli orcidid: 0000-0002-3238-5430 surname: Shao fullname: Shao, Yanli email: shaoyanli@hdu.edu.cn organization: School of Computer Science and Technology, Hangzhou Dianzi University, China |
| BookMark | eNqFkM1KAzEUhYMo2Kpv4GKWbma8yfxlXAhS1AqiILoOaXIHU6eTepNW3Pkavp5P4pS6cqGrsznfgfON2W7ve2TsmEPGgVen86zHVY8xEyB4BjKDXOywEZd1k4pail02AtnkaQUS9tk4hDkAVLLIR2z6gGZFhH1Mhg3S3RDxzdNL0pJfJNpapK-Pz5AskcISTXRrPEsmmug97bx_0c-obfJwd3fI9lrdBTz6yQP2dHX5OJmmt_fXN5OL29TkeRNTXhYzAFM0Vc3rFgrBQcykldWMt20DvNFosTYSmqrUra2KAtqytjOh85KjxvyAnWx3l-RfVxiiWrhgsOt0j34VlCjrcvjGJQzVs23VkA-BsFXGRR2d7yNp1ykOaqNPzdVWn9roUyDVoG-Ai1_wktxC0_t_2PkWw8HB2iGpYBz2Bq2jwZ6y3v098A0tco8t |
| CitedBy_id | crossref_primary_10_1177_14759217241243353 crossref_primary_10_1016_j_apt_2024_104743 crossref_primary_10_1109_ACCESS_2025_3566005 crossref_primary_10_3390_a17110489 crossref_primary_10_1016_j_measurement_2022_112302 crossref_primary_10_1016_j_aeue_2025_155791 crossref_primary_10_1049_tje2_70100 crossref_primary_10_1088_1361_6501_ace644 crossref_primary_10_1109_TPEL_2024_3459470 crossref_primary_10_3390_rs17152737 crossref_primary_10_1016_j_applthermaleng_2024_124693 crossref_primary_10_1016_j_engappai_2024_109400 crossref_primary_10_3390_app13042545 crossref_primary_10_1016_j_measurement_2025_117442 crossref_primary_10_1093_ijlct_ctae139 crossref_primary_10_3390_s22207835 crossref_primary_10_3724_j_issn_1007_2802_20240156 crossref_primary_10_1007_s40808_025_02353_8 crossref_primary_10_1016_j_aei_2022_101856 crossref_primary_10_1016_j_energy_2024_130326 crossref_primary_10_1016_j_imu_2023_101428 crossref_primary_10_3233_JIFS_230528 crossref_primary_10_32604_cmc_2025_060567 crossref_primary_10_1080_19475705_2022_2102942 crossref_primary_10_1016_j_agwat_2024_108972 crossref_primary_10_1016_j_jclepro_2022_134048 |
| Cites_doi | 10.4249/scholarpedia.5947 10.1007/s00371-019-01774-8 10.1038/323533a0 10.1162/neco.1997.9.8.1735 10.1109/5.726791 10.1109/CVPR.2017.113 10.1109/TNNLS.2016.2582924 10.1109/CVPR.2017.683 10.1117/1.JEI.27.3.033002 10.1109/TIP.2020.2965989 10.1007/s10617-019-09220-7 10.1109/CVPR.2015.7298594 10.1109/TGRS.2017.2755542 10.1162/neco.1989.1.4.541 10.1016/j.future.2017.09.073 10.1109/CVPR.2016.90 10.1007/s41095-020-0185-5 |
| ContentType | Journal Article |
| Copyright | 2021 Elsevier Ltd Copyright © 2021 Elsevier Ltd. All rights reserved. |
| Copyright_xml | – notice: 2021 Elsevier Ltd – notice: Copyright © 2021 Elsevier Ltd. All rights reserved. |
| DBID | AAYXX CITATION 7X8 |
| DOI | 10.1016/j.neunet.2021.08.032 |
| DatabaseName | CrossRef MEDLINE - Academic |
| DatabaseTitle | CrossRef MEDLINE - Academic |
| DatabaseTitleList | MEDLINE - Academic |
| Database_xml | – sequence: 1 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISSN | 1879-2782 |
| EndPage | 306 |
| ExternalDocumentID | 10_1016_j_neunet_2021_08_032 S0893608021003452 |
| GroupedDBID | --- --K --M -~X .DC .~1 0R~ 123 186 1B1 1RT 1~. 1~5 29N 4.4 457 4G. 53G 5RE 5VS 6TJ 7-5 71M 8P~ 9JM 9JN AABNK AACTN AADPK AAEDT AAEDW AAIAV AAIKJ AAKOC AALRI AAOAW AAQFI AAQXK AAXLA AAXUO AAYFN ABAOU ABBOA ABCQJ ABEFU ABFNM ABFRF ABHFT ABIVO ABJNI ABLJU ABMAC ABXDB ABYKQ ACAZW ACDAQ ACGFO ACGFS ACIUM ACNNM ACRLP ACZNC ADBBV ADEZE ADGUI ADJOM ADMUD ADRHT AEBSH AECPX AEFWE AEKER AENEX AFKWA AFTJW AFXIZ AGHFR AGUBO AGWIK AGYEJ AHHHB AHJVU AHZHX AIALX AIEXJ AIKHN AITUG AJBFU AJOXV ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AOUOD ARUGR ASPBG AVWKF AXJTR AZFZN BJAXD BKOJK BLXMC CS3 DU5 EBS EFJIC EFLBG EJD EO8 EO9 EP2 EP3 F0J F5P FDB FEDTE FGOYB FIRID FNPLU FYGXN G-2 G-Q G8K GBLVA GBOLZ HLZ HMQ HVGLF HZ~ IHE J1W JJJVA K-O KOM KZ1 LG9 LMP M2V M41 MHUIS MO0 MOBAO MVM N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. Q38 R2- RIG ROL RPZ SBC SCC SDF SDG SDP SES SEW SNS SPC SPCBC SSN SST SSV SSW SSZ T5K TAE UAP UNMZH VOH WUQ XPP ZMT ~G- 9DU AATTM AAXKI AAYWO AAYXX ABDPE ABWVN ACLOT ACRPL ACVFH ADCNI ADNMO AEIPS AEUPX AFJKZ AFPUW AGQPQ AIGII AIIUN AKBMS AKRWK AKYEP ANKPU APXCP CITATION EFKBS ~HD 7X8 |
| ID | FETCH-LOGICAL-c339t-154b00c496717f042102b8d86b1ff9019aede7c80965afd6440f57db2a351eae3 |
| ISICitedReferencesCount | 30 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000704005300018&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0893-6080 1879-2782 |
| IngestDate | Mon Sep 29 02:49:40 EDT 2025 Tue Nov 18 21:08:39 EST 2025 Sat Nov 29 07:19:00 EST 2025 Fri Feb 23 02:41:14 EST 2024 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Deep learning Sequence modeling Parallel computation Carry-lookahead |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c339t-154b00c496717f042102b8d86b1ff9019aede7c80965afd6440f57db2a351eae3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
| ORCID | 0000-0003-1208-972X 0000-0002-5619-5252 0000-0002-9880-0342 0000-0002-3238-5430 |
| PQID | 2575068180 |
| PQPubID | 23479 |
| PageCount | 10 |
| ParticipantIDs | proquest_miscellaneous_2575068180 crossref_citationtrail_10_1016_j_neunet_2021_08_032 crossref_primary_10_1016_j_neunet_2021_08_032 elsevier_sciencedirect_doi_10_1016_j_neunet_2021_08_032 |
| PublicationCentury | 2000 |
| PublicationDate | December 2021 2021-12-00 20211201 |
| PublicationDateYYYYMMDD | 2021-12-01 |
| PublicationDate_xml | – month: 12 year: 2021 text: December 2021 |
| PublicationDecade | 2020 |
| PublicationTitle | Neural networks |
| PublicationYear | 2021 |
| Publisher | Elsevier Ltd |
| Publisher_xml | – name: Elsevier Ltd |
| References | Zhang, Wu, Che, Lin, Memisevic, Salakhutdinov (b51) 2016 Graves (b10) 2013 Cho, Van Merriënboer, Gulcehre, Bahdanau, Bougares, Schwenk (b7) 2014 Pascanu, Gulcehre, Cho, Bengio (b37) 2013 Hochreiter, Schmidhuber (b14) 1997; 9 Bahdanau, Cho, Bengio (b2) 2014 Goodfellow, Bengio, Courville, Bengio (b9) 2016 Shazeer, Mirhoseini, Maziarz, Davis, Le, Hinton (b40) 2017 Krueger, Maharaj, Kramár, Pezeshki, Ballas, Ke (b23) 2016 Vohra, Goel, Sahoo (b45) 2015 Lea, C., Flynn, M. D., Vidal, R., Reiter, A., & Hager, G. D. (2017). Temporal convolutional networks for action segmentation and detection. In Jing, Shen, Dubcek, Peurifoy, Skirlo, LeCun (b16) 2017 LeCun, Boser, Denker, Henderson, Howard, Hubbard (b29) 1989; 1 Rosenberger (b38) 1960 (pp. 156–165). Sutskever, Vinyals, Le (b42) 2014 Jozefowicz, Zaremba, Sutskever (b18) 2015 Rumelhart, Hinton, Williams (b39) 1986; 323 LeCun, Bottou, Bengio, Haffner (b30) 1998; 86 Zhang, He, Luo, Xu, He (b50) 2020; 6 Bai, Kolter, Koltun (b3) 2018 Chang, Zhang, Han, Yu, Guo, Tan (b5) 2017; 30 Kirichenko, Vernik, Kamkar, Walter, Miller, Albu (b21) 2019; 29 Miyamoto, Cho (b35) 2016 Shi, Chen, Wang, Yeung, Wong, Woo (b41) 2015; 2015 He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In (pp. 3156–3164). Greff, Srivastava, Koutník, Steunebrink, Schmidhuber (b11) 2016; 28 Krueger, Memisevic (b24) 2015 LeCun, Kanter, Solla (b31) 1991 Wu, Schuster, Chen, Le, Norouzi, Macherey (b48) 2016 Le, Jaitly, Hinton (b26) 2015 Kalchbrenner, Espeholt, Simonyan, Oord, Graves, Kavukcuoglu (b19) 2016 Jozefowicz, Vinyals, Schuster, Shazeer, Wu (b17) 2016 Allan, Williams (b1) 2005; 17 Zhong, Li, Luo, Chapman (b52) 2017; 56 Chung, Gulcehre, Cho, Bengio (b8) 2014 Le, Mikolov (b27) 2014 Zhang, He (b49) 2020; 36 Hinton (b13) 2009; 4 Mikolov, T., Sutskever, I., Deoras, A., Le, H.-S., Kombrink, S., & Cernocky, J. (2012). Subword language modeling with neural networks, preprint (pp. 770–778). Krizhevsky, Sutskever, Hinton (b22) 2012; 25 Zhou, He, Qiu (b54) 2018; 27 Chen, Tan, Wang, Lu, Hu, Fu (b6) 2020; 29 Wang, F., Jiang, M., Qian, C., Yang, S., Li, C., & Zhang, H., et al. (2017). Residual attention network for image classification. In Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., & Anguelov, D., et al. (2015). Going deeper with convolutions. In 8, 67. Kuchaiev, Ginsburg (b25) 2017 Oord, Dieleman, Zen, Simonyan, Vinyals, Graves (b36) 2016 (pp. 1–9). Marcus, Santorini, Marcinkiewicz (b32) 1993; 19 Bradbury, Merity, Xiong, Socher (b4) 2016 Wisdom, Powers, Hershey, Roux, Atlas (b47) 2016 Thapliyal, Srinivas (b44) 2005 Merity, Keskar, Socher (b33) 2017 Zhou, He, Hou, Qiu (b53) 2018; 79 Hou, Yan, He (b15) 2019; 23 Kalchbrenner, Grefenstette, Blunsom (b20) 2014 Kuchaiev (10.1016/j.neunet.2021.08.032_b25) 2017 Krueger (10.1016/j.neunet.2021.08.032_b24) 2015 Bai (10.1016/j.neunet.2021.08.032_b3) 2018 10.1016/j.neunet.2021.08.032_b46 Greff (10.1016/j.neunet.2021.08.032_b11) 2016; 28 Jozefowicz (10.1016/j.neunet.2021.08.032_b18) 2015 Bahdanau (10.1016/j.neunet.2021.08.032_b2) 2014 Chen (10.1016/j.neunet.2021.08.032_b6) 2020; 29 10.1016/j.neunet.2021.08.032_b43 Wu (10.1016/j.neunet.2021.08.032_b48) 2016 Chung (10.1016/j.neunet.2021.08.032_b8) 2014 Sutskever (10.1016/j.neunet.2021.08.032_b42) 2014 Zhou (10.1016/j.neunet.2021.08.032_b54) 2018; 27 Krizhevsky (10.1016/j.neunet.2021.08.032_b22) 2012; 25 Pascanu (10.1016/j.neunet.2021.08.032_b37) 2013 Hinton (10.1016/j.neunet.2021.08.032_b13) 2009; 4 Krueger (10.1016/j.neunet.2021.08.032_b23) 2016 Zhang (10.1016/j.neunet.2021.08.032_b51) 2016 Goodfellow (10.1016/j.neunet.2021.08.032_b9) 2016 Cho (10.1016/j.neunet.2021.08.032_b7) 2014 10.1016/j.neunet.2021.08.032_b12 Hou (10.1016/j.neunet.2021.08.032_b15) 2019; 23 Shazeer (10.1016/j.neunet.2021.08.032_b40) 2017 Allan (10.1016/j.neunet.2021.08.032_b1) 2005; 17 Merity (10.1016/j.neunet.2021.08.032_b33) 2017 Zhang (10.1016/j.neunet.2021.08.032_b50) 2020; 6 LeCun (10.1016/j.neunet.2021.08.032_b29) 1989; 1 Zhou (10.1016/j.neunet.2021.08.032_b53) 2018; 79 Bradbury (10.1016/j.neunet.2021.08.032_b4) 2016 Rosenberger (10.1016/j.neunet.2021.08.032_b38) 1960 Chang (10.1016/j.neunet.2021.08.032_b5) 2017; 30 10.1016/j.neunet.2021.08.032_b28 Zhong (10.1016/j.neunet.2021.08.032_b52) 2017; 56 Kalchbrenner (10.1016/j.neunet.2021.08.032_b20) 2014 Marcus (10.1016/j.neunet.2021.08.032_b32) 1993; 19 Oord (10.1016/j.neunet.2021.08.032_b36) 2016 Le (10.1016/j.neunet.2021.08.032_b27) 2014 Rumelhart (10.1016/j.neunet.2021.08.032_b39) 1986; 323 Kirichenko (10.1016/j.neunet.2021.08.032_b21) 2019; 29 Thapliyal (10.1016/j.neunet.2021.08.032_b44) 2005 Jing (10.1016/j.neunet.2021.08.032_b16) 2017 Wisdom (10.1016/j.neunet.2021.08.032_b47) 2016 LeCun (10.1016/j.neunet.2021.08.032_b30) 1998; 86 LeCun (10.1016/j.neunet.2021.08.032_b31) 1991 Shi (10.1016/j.neunet.2021.08.032_b41) 2015; 2015 Graves (10.1016/j.neunet.2021.08.032_b10) 2013 Kalchbrenner (10.1016/j.neunet.2021.08.032_b19) 2016 Hochreiter (10.1016/j.neunet.2021.08.032_b14) 1997; 9 Le (10.1016/j.neunet.2021.08.032_b26) 2015 Miyamoto (10.1016/j.neunet.2021.08.032_b35) 2016 Zhang (10.1016/j.neunet.2021.08.032_b49) 2020; 36 10.1016/j.neunet.2021.08.032_b34 Jozefowicz (10.1016/j.neunet.2021.08.032_b17) 2016 Vohra (10.1016/j.neunet.2021.08.032_b45) 2015 |
| References_xml | – reference: He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In – year: 2014 ident: b42 article-title: Sequence to sequence learning with neural networks – year: 2018 ident: b3 article-title: An empirical evaluation of generic convolutional and recurrent networks for sequence modeling – volume: 17 start-page: 25 year: 2005 end-page: 32 ident: b1 article-title: Harmonising chorales by probabilistic inference publication-title: Advances in Neural Information Processing Systems – year: 2015 ident: b26 article-title: A simple way to initialize recurrent networks of rectified linear units – year: 2016 ident: b36 article-title: WaveNet: A generative model for raw audio – year: 2014 ident: b8 article-title: Empirical evaluation of gated recurrent neural networks on sequence modeling – volume: 29 start-page: 1 year: 2019 end-page: 7 ident: b21 article-title: ERSFQ 8-bit parallel arithmetic logic unit publication-title: IEEE Transactions on Applied Superconductivity – start-page: 805 year: 2005 end-page: 817 ident: b44 article-title: A novel reversible TSG gate and its application for designing reversible carry look-ahead and other adder architectures publication-title: Asia-Pacific conference on advances in computer systems architecture – year: 2013 ident: b10 article-title: Generating sequences with recurrent neural networks – year: 2016 ident: b35 article-title: Gated word-character recurrent language model – reference: Wang, F., Jiang, M., Qian, C., Yang, S., Li, C., & Zhang, H., et al. (2017). Residual attention network for image classification. In – year: 2014 ident: b2 article-title: Neural machine translation by jointly learning to align and translate – volume: 2015 start-page: 802 year: 2015 end-page: 810 ident: b41 article-title: Convolutional LSTM network: A machine learning approach for precipitation nowcasting publication-title: Advances in Neural Information Processing Systems – year: 1960 ident: b38 article-title: Simultaneous carry adder – start-page: 1 year: 2015 end-page: 4 ident: b45 article-title: Modeling temporal dependencies in data using a DBN-LSTM publication-title: IEEE international conference on data science and advanced analytics – volume: 9 start-page: 1735 year: 1997 end-page: 1780 ident: b14 article-title: Long short-term memory publication-title: Neural Computation – reference: (pp. 3156–3164). – reference: Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., & Anguelov, D., et al. (2015). Going deeper with convolutions. In – year: 2016 ident: b51 article-title: Architectural complexity measures of recurrent neural networks – year: 2017 ident: b33 article-title: Regularizing and optimizing LSTM language models – volume: 6 start-page: 455 year: 2020 end-page: 466 ident: b50 article-title: Weight asynchronous update: Improving the diversity of filters in a deep convolutional network publication-title: Computational Visual Media – reference: (pp. 156–165). – start-page: 918 year: 1991 end-page: 924 ident: b31 article-title: Second order properties of error surfaces: Learning time and generalization publication-title: Advances in neural information processing systems, Vol. 3 – reference: ) 8, 67. – start-page: 2342 year: 2015 end-page: 2350 ident: b18 article-title: An empirical exploration of recurrent network architectures publication-title: International conference on machine learning – volume: 4 start-page: 5947 year: 2009 ident: b13 article-title: Deep belief networks publication-title: Scholarpedia – volume: 56 start-page: 847 year: 2017 end-page: 858 ident: b52 article-title: Spectral–spatial residual network for hyperspectral image classification: A 3-D deep learning framework publication-title: IEEE Transactions on Geoscience and Remote Sensing – volume: 30 start-page: 77 year: 2017 end-page: 87 ident: b5 article-title: Dilated recurrent neural networks publication-title: Advances in Neural Information Processing Systems – year: 2016 ident: b9 article-title: Deep learning, Vol. 1 – volume: 28 start-page: 2222 year: 2016 end-page: 2232 ident: b11 article-title: LSTM: A search space odyssey publication-title: IEEE Transactions on Neural Networks and Learning Systems – volume: 23 start-page: 57 year: 2019 end-page: 77 ident: b15 article-title: A survey on partitioning models, solution algorithms and algorithm parallelization for hardware/software co-design publication-title: Design Automation for Embedded System – volume: 1 start-page: 541 year: 1989 end-page: 551 ident: b29 article-title: Backpropagation applied to handwritten zip code recognition publication-title: Neural Computation – year: 2016 ident: b23 article-title: Zoneout: Regularizing RNNs by randomly preserving hidden activations – start-page: 1733 year: 2017 end-page: 1741 ident: b16 article-title: Tunable efficient unitary neural networks (EUNN) and their application to RNNs publication-title: International conference on machine learning – volume: 25 start-page: 1097 year: 2012 end-page: 1105 ident: b22 article-title: ImageNet classification with deep convolutional neural networks publication-title: Advances in Neural Information Processing Systems – year: 2016 ident: b4 article-title: Quasi-recurrent neural networks – year: 2013 ident: b37 article-title: How to construct deep recurrent neural networks – reference: (pp. 1–9). – volume: 79 start-page: 473 year: 2018 end-page: 487 ident: b53 article-title: Parallel ant colony optimization on multi-core SIMD CPUs publication-title: Future Generation Computer Systems – volume: 36 start-page: 1797 year: 2020 end-page: 1808 ident: b49 article-title: DRCDN: learning deep residual convolutional dehazing networks publication-title: The Visual Computer – reference: Lea, C., Flynn, M. D., Vidal, R., Reiter, A., & Hager, G. D. (2017). Temporal convolutional networks for action segmentation and detection. In – volume: 86 start-page: 2278 year: 1998 end-page: 2324 ident: b30 article-title: Gradient-based learning applied to document recognition publication-title: Proceedings of the IEEE – reference: (pp. 770–778). – volume: 19 start-page: 313 year: 1993 end-page: 330 ident: b32 article-title: Building a large annotated corpus of English: The Penn Treebank publication-title: Computational Linguistics – year: 2014 ident: b20 article-title: A convolutional neural network for modelling sentences – year: 2016 ident: b19 article-title: Neural machine translation in linear time – year: 2017 ident: b40 article-title: Outrageously large neural networks: The sparsely-gated mixture-of-experts layer – year: 2014 ident: b7 article-title: Learning phrase representations using RNN encoder-decoder for statistical machine translation – year: 2015 ident: b24 article-title: Regularizing RNNs by stabilizing activations – volume: 323 start-page: 533 year: 1986 end-page: 536 ident: b39 article-title: Learning representations by back-propagating errors publication-title: Nature – volume: 27 year: 2018 ident: b54 article-title: Accelerating image convolution filtering algorithms on integrated CPU–GPU architectures publication-title: Journal of Electronic Imaging – reference: Mikolov, T., Sutskever, I., Deoras, A., Le, H.-S., Kombrink, S., & Cernocky, J. (2012). Subword language modeling with neural networks, preprint ( – start-page: 1188 year: 2014 end-page: 1196 ident: b27 article-title: Distributed representations of sentences and documents publication-title: International conference on machine learning – year: 2017 ident: b25 article-title: Factorization tricks for LSTM networks – year: 2016 ident: b48 article-title: Google’s neural machine translation system: Bridging the gap between human and machine translation – volume: 29 start-page: 3763 year: 2020 end-page: 3776 ident: b6 article-title: Reverse attention-based residual network for salient object detection publication-title: IEEE Transactions on Image Processing – year: 2016 ident: b17 article-title: Exploring the limits of language modeling – year: 2016 ident: b47 article-title: Full-capacity unitary recurrent neural networks – year: 2016 ident: 10.1016/j.neunet.2021.08.032_b4 – volume: 4 start-page: 5947 issue: 5 year: 2009 ident: 10.1016/j.neunet.2021.08.032_b13 article-title: Deep belief networks publication-title: Scholarpedia doi: 10.4249/scholarpedia.5947 – volume: 36 start-page: 1797 issue: 9 year: 2020 ident: 10.1016/j.neunet.2021.08.032_b49 article-title: DRCDN: learning deep residual convolutional dehazing networks publication-title: The Visual Computer doi: 10.1007/s00371-019-01774-8 – year: 2016 ident: 10.1016/j.neunet.2021.08.032_b19 – year: 1960 ident: 10.1016/j.neunet.2021.08.032_b38 – volume: 19 start-page: 313 issue: 2 year: 1993 ident: 10.1016/j.neunet.2021.08.032_b32 article-title: Building a large annotated corpus of English: The Penn Treebank publication-title: Computational Linguistics – volume: 323 start-page: 533 issue: 6088 year: 1986 ident: 10.1016/j.neunet.2021.08.032_b39 article-title: Learning representations by back-propagating errors publication-title: Nature doi: 10.1038/323533a0 – year: 2014 ident: 10.1016/j.neunet.2021.08.032_b7 – year: 2016 ident: 10.1016/j.neunet.2021.08.032_b9 – year: 2016 ident: 10.1016/j.neunet.2021.08.032_b47 – start-page: 2342 year: 2015 ident: 10.1016/j.neunet.2021.08.032_b18 article-title: An empirical exploration of recurrent network architectures – volume: 9 start-page: 1735 issue: 8 year: 1997 ident: 10.1016/j.neunet.2021.08.032_b14 article-title: Long short-term memory publication-title: Neural Computation doi: 10.1162/neco.1997.9.8.1735 – volume: 86 start-page: 2278 issue: 11 year: 1998 ident: 10.1016/j.neunet.2021.08.032_b30 article-title: Gradient-based learning applied to document recognition publication-title: Proceedings of the IEEE doi: 10.1109/5.726791 – year: 2016 ident: 10.1016/j.neunet.2021.08.032_b51 – ident: 10.1016/j.neunet.2021.08.032_b28 doi: 10.1109/CVPR.2017.113 – volume: 29 start-page: 1 issue: 5 year: 2019 ident: 10.1016/j.neunet.2021.08.032_b21 article-title: ERSFQ 8-bit parallel arithmetic logic unit publication-title: IEEE Transactions on Applied Superconductivity – year: 2013 ident: 10.1016/j.neunet.2021.08.032_b37 – start-page: 805 year: 2005 ident: 10.1016/j.neunet.2021.08.032_b44 article-title: A novel reversible TSG gate and its application for designing reversible carry look-ahead and other adder architectures – volume: 28 start-page: 2222 issue: 10 year: 2016 ident: 10.1016/j.neunet.2021.08.032_b11 article-title: LSTM: A search space odyssey publication-title: IEEE Transactions on Neural Networks and Learning Systems doi: 10.1109/TNNLS.2016.2582924 – year: 2016 ident: 10.1016/j.neunet.2021.08.032_b17 – ident: 10.1016/j.neunet.2021.08.032_b46 doi: 10.1109/CVPR.2017.683 – volume: 30 start-page: 77 year: 2017 ident: 10.1016/j.neunet.2021.08.032_b5 article-title: Dilated recurrent neural networks publication-title: Advances in Neural Information Processing Systems – volume: 17 start-page: 25 year: 2005 ident: 10.1016/j.neunet.2021.08.032_b1 article-title: Harmonising chorales by probabilistic inference publication-title: Advances in Neural Information Processing Systems – volume: 27 issue: 3 year: 2018 ident: 10.1016/j.neunet.2021.08.032_b54 article-title: Accelerating image convolution filtering algorithms on integrated CPU–GPU architectures publication-title: Journal of Electronic Imaging doi: 10.1117/1.JEI.27.3.033002 – year: 2014 ident: 10.1016/j.neunet.2021.08.032_b8 – volume: 29 start-page: 3763 year: 2020 ident: 10.1016/j.neunet.2021.08.032_b6 article-title: Reverse attention-based residual network for salient object detection publication-title: IEEE Transactions on Image Processing doi: 10.1109/TIP.2020.2965989 – volume: 25 start-page: 1097 year: 2012 ident: 10.1016/j.neunet.2021.08.032_b22 article-title: ImageNet classification with deep convolutional neural networks publication-title: Advances in Neural Information Processing Systems – volume: 23 start-page: 57 issue: 1 year: 2019 ident: 10.1016/j.neunet.2021.08.032_b15 article-title: A survey on partitioning models, solution algorithms and algorithm parallelization for hardware/software co-design publication-title: Design Automation for Embedded System doi: 10.1007/s10617-019-09220-7 – year: 2017 ident: 10.1016/j.neunet.2021.08.032_b25 – year: 2016 ident: 10.1016/j.neunet.2021.08.032_b48 – year: 2016 ident: 10.1016/j.neunet.2021.08.032_b36 – year: 2013 ident: 10.1016/j.neunet.2021.08.032_b10 – year: 2014 ident: 10.1016/j.neunet.2021.08.032_b2 – volume: 2015 start-page: 802 year: 2015 ident: 10.1016/j.neunet.2021.08.032_b41 article-title: Convolutional LSTM network: A machine learning approach for precipitation nowcasting publication-title: Advances in Neural Information Processing Systems – start-page: 918 year: 1991 ident: 10.1016/j.neunet.2021.08.032_b31 article-title: Second order properties of error surfaces: Learning time and generalization – year: 2014 ident: 10.1016/j.neunet.2021.08.032_b42 – start-page: 1 year: 2015 ident: 10.1016/j.neunet.2021.08.032_b45 article-title: Modeling temporal dependencies in data using a DBN-LSTM – ident: 10.1016/j.neunet.2021.08.032_b43 doi: 10.1109/CVPR.2015.7298594 – year: 2018 ident: 10.1016/j.neunet.2021.08.032_b3 – volume: 56 start-page: 847 issue: 2 year: 2017 ident: 10.1016/j.neunet.2021.08.032_b52 article-title: Spectral–spatial residual network for hyperspectral image classification: A 3-D deep learning framework publication-title: IEEE Transactions on Geoscience and Remote Sensing doi: 10.1109/TGRS.2017.2755542 – volume: 1 start-page: 541 issue: 4 year: 1989 ident: 10.1016/j.neunet.2021.08.032_b29 article-title: Backpropagation applied to handwritten zip code recognition publication-title: Neural Computation doi: 10.1162/neco.1989.1.4.541 – volume: 79 start-page: 473 year: 2018 ident: 10.1016/j.neunet.2021.08.032_b53 article-title: Parallel ant colony optimization on multi-core SIMD CPUs publication-title: Future Generation Computer Systems doi: 10.1016/j.future.2017.09.073 – year: 2017 ident: 10.1016/j.neunet.2021.08.032_b40 – start-page: 1733 year: 2017 ident: 10.1016/j.neunet.2021.08.032_b16 article-title: Tunable efficient unitary neural networks (EUNN) and their application to RNNs – year: 2016 ident: 10.1016/j.neunet.2021.08.032_b35 – start-page: 1188 year: 2014 ident: 10.1016/j.neunet.2021.08.032_b27 article-title: Distributed representations of sentences and documents – ident: 10.1016/j.neunet.2021.08.032_b34 – year: 2015 ident: 10.1016/j.neunet.2021.08.032_b24 – ident: 10.1016/j.neunet.2021.08.032_b12 doi: 10.1109/CVPR.2016.90 – volume: 6 start-page: 455 issue: 4 year: 2020 ident: 10.1016/j.neunet.2021.08.032_b50 article-title: Weight asynchronous update: Improving the diversity of filters in a deep convolutional network publication-title: Computational Visual Media doi: 10.1007/s41095-020-0185-5 – year: 2016 ident: 10.1016/j.neunet.2021.08.032_b23 – year: 2017 ident: 10.1016/j.neunet.2021.08.032_b33 – year: 2014 ident: 10.1016/j.neunet.2021.08.032_b20 – year: 2015 ident: 10.1016/j.neunet.2021.08.032_b26 |
| SSID | ssj0006843 |
| Score | 2.5694387 |
| Snippet | The recurrent network architecture is a widely used model in sequence modeling, but its serial dependency hinders the computation parallelization, which makes... |
| SourceID | proquest crossref elsevier |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 297 |
| SubjectTerms | Carry-lookahead Deep learning Parallel computation Sequence modeling |
| Title | Recurrent neural network from adder’s perspective: Carry-lookahead RNN |
| URI | https://dx.doi.org/10.1016/j.neunet.2021.08.032 https://www.proquest.com/docview/2575068180 |
| Volume | 144 |
| WOSCitedRecordID | wos000704005300018&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: ScienceDirect database customDbUrl: eissn: 1879-2782 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0006843 issn: 0893-6080 databaseCode: AIEXJ dateStart: 19950101 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1bb9MwFLa6jQdeYNzEGExGQvCAXK1JGtt7m6pWA1UBjQ6FJ8uJHSlTlZa2u_DvdxzbaXZBGw-8RJUVJ-05p58_H58LQh9qkh7xjITKeKviQhHGOOx5Cp5LBQyV1om0P8c0SVia8u-dzg-fC3M-pVXFLi_5_L-qGsZA2SZ19h_U3TwUBuAzKB2uoHa4Pkjxx8aDXtdcMrUqQQOVjfS2mSTSVgyhS1Ow2KdZGrfAQC4Wf8gUOLcEfFafj5OkTVyTa89axx2Wzt98JGcXumzcqLYywUiXrcGBtKc8ZbXGYzv518ytn879EPRaoRwWMRnlJKC2g1BX3zHmYTaK2kBpo3JvAbj1JZx2QULwi7rmjXWJVecEvVYvO_kmRifjsZgM08nH-W9iWomZI3fXV2UDbQW0zwHqtg6_DNOvzQIdM5t34b-lz6isw_5uv_hvjOXG2l0Tksk2euJ2EvjQWsAz1NHVc_TUd-nADrRfoFFjENgaBHZKxMYgcG0Qn5a4ZQ4H-IYxYDCGl-hkNJwMjojrnkHyMOQrAtwYIDWPeAw79gKwGahkxhSLs15RAAvkUitNc2bK_8hCAS_eL_pUZYEM-z0tdfgKbVazSr9GWPOY8T7tRSoEMmMaLgQ8y2mcZcE-p0rvoNBLSOSutLzpcDIVPobwVFi5CiNXYRqfhsEOIs2suS2tcs_91AtfOHpoaZ8A47ln5nuvKwHoaY7EZKVnZ0sRGCyKTb2DNw-4Zxc9Xv8F3qLN1eJMv0OP8vNVuVzsoQ2asj1nbFc5v5Bh |
| linkProvider | Elsevier |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Recurrent+neural+network+from+adder%27s+perspective%3A+Carry-lookahead+RNN&rft.jtitle=Neural+networks&rft.au=Jiang%2C+Haowei&rft.au=Qin%2C+Feiwei&rft.au=Cao%2C+Jin&rft.au=Peng%2C+Yong&rft.date=2021-12-01&rft.issn=1879-2782&rft.eissn=1879-2782&rft.volume=144&rft.spage=297&rft_id=info:doi/10.1016%2Fj.neunet.2021.08.032&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0893-6080&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0893-6080&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0893-6080&client=summon |