Performance Impacts of Analog ReRAM Non-ideality on Neuromorphic Computing

Resistive random access memory (ReRAM) is often considered as a strong candidate for storing the weights in non-von Neumann neuromorphic computing systems. This paper studies how nonideal memory characteristics, which include programing error, read fluctuation, and retention, affect the inference ac...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	IEEE transactions on electron devices Ročník 66; číslo 3; s. 1289 - 1295
Hlavní autoři:	Lin, Yu-Hsuan, Wang, Chao-Hung, Lee, Ming-Hsiu, Lee, Dai-Ying, Lin, Yu-Yu, Lee, Feng-Min, Lung, Hsiang-Lan, Wang, Keh-Chung, Tseng, Tseung-Yuen, Lu, Chih-Yuan
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	New York IEEE 01.03.2019 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:	Accuracy Analog memory Artificial neural networks Computation Computer simulation Error reduction Inference Mapping Memory management Multilayers Neural networks Neuromorphic computing Neuromorphics noise Random access memory reliability Resistance resistive random access memory (ReRAM) stability Synapses Training Variation Weight
ISSN:	0018-9383, 1557-9646
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Abstract	Resistive random access memory (ReRAM) is often considered as a strong candidate for storing the weights in non-von Neumann neuromorphic computing systems. This paper studies how nonideal memory characteristics, which include programing error, read fluctuation, and retention, affect the inference accuracy of the analog ReRAM neural networks by incorporating memory characteristics extracted from 1-Mb ReRAM into a simulated inference-only neural network. This paper also shows that the different layer in the network can tolerate different amount of such imperfects. We learned four key points: 1) the conductance range of memory with less relative fluctuation is preferred for designing the weight-conductance mapping; 2) the control of programing error is essential for high inference accuracy; 3) retention-induced conductance drift can be fatal to the neuromorphic system. A compensation scheme is proposed in this paper which can effectively recover the inference accuracy; and 4) for multilayer networks, avoiding weight errors in the front layers can help to maintain the inference accuracy by reducing calculation error which may otherwise accumulate and pass down the networks. The concepts and approaches of this paper can also be applied to evaluate other types of nonvolatile memories for artificial neural networks.
AbstractList	Resistive random access memory (ReRAM) is often considered as a strong candidate for storing the weights in non-von Neumann neuromorphic computing systems. This paper studies how nonideal memory characteristics, which include programing error, read fluctuation, and retention, affect the inference accuracy of the analog ReRAM neural networks by incorporating memory characteristics extracted from 1-Mb ReRAM into a simulated inference-only neural network. This paper also shows that the different layer in the network can tolerate different amount of such imperfects. We learned four key points: 1) the conductance range of memory with less relative fluctuation is preferred for designing the weight-conductance mapping; 2) the control of programing error is essential for high inference accuracy; 3) retention-induced conductance drift can be fatal to the neuromorphic system. A compensation scheme is proposed in this paper which can effectively recover the inference accuracy; and 4) for multilayer networks, avoiding weight errors in the front layers can help to maintain the inference accuracy by reducing calculation error which may otherwise accumulate and pass down the networks. The concepts and approaches of this paper can also be applied to evaluate other types of nonvolatile memories for artificial neural networks.
Author	Lin, Yu-Yu Lung, Hsiang-Lan Lu, Chih-Yuan Lee, Ming-Hsiu Lin, Yu-Hsuan Wang, Keh-Chung Wang, Chao-Hung Tseng, Tseung-Yuen Lee, Dai-Ying Lee, Feng-Min
Author_xml	– sequence: 1 givenname: Yu-Hsuan orcidid: 0000-0001-8352-3584 surname: Lin fullname: Lin, Yu-Hsuan email: yhlin.ee01g@nctu.edu.tw organization: Department of Electronics Engineering, Institute of Electronics, National Chiao Tung University, Hsinchu, Taiwan – sequence: 2 givenname: Chao-Hung surname: Wang fullname: Wang, Chao-Hung organization: Emerging Central Laboratory, Macronix International Co., Ltd., Hsinchu, Taiwan – sequence: 3 givenname: Ming-Hsiu surname: Lee fullname: Lee, Ming-Hsiu organization: Emerging Central Laboratory, Macronix International Co., Ltd., Hsinchu, Taiwan – sequence: 4 givenname: Dai-Ying surname: Lee fullname: Lee, Dai-Ying organization: Emerging Central Laboratory, Macronix International Co., Ltd., Hsinchu, Taiwan – sequence: 5 givenname: Yu-Yu surname: Lin fullname: Lin, Yu-Yu organization: Emerging Central Laboratory, Macronix International Co., Ltd., Hsinchu, Taiwan – sequence: 6 givenname: Feng-Min surname: Lee fullname: Lee, Feng-Min organization: Emerging Central Laboratory, Macronix International Co., Ltd., Hsinchu, Taiwan – sequence: 7 givenname: Hsiang-Lan surname: Lung fullname: Lung, Hsiang-Lan organization: Emerging Central Laboratory, Macronix International Co., Ltd., Hsinchu, Taiwan – sequence: 8 givenname: Keh-Chung surname: Wang fullname: Wang, Keh-Chung organization: Emerging Central Laboratory, Macronix International Co., Ltd., Hsinchu, Taiwan – sequence: 9 givenname: Tseung-Yuen orcidid: 0000-0003-1158-5289 surname: Tseng fullname: Tseng, Tseung-Yuen organization: Department of Electronics Engineering, Institute of Electronics, National Chiao Tung University, Hsinchu, Taiwan – sequence: 10 givenname: Chih-Yuan surname: Lu fullname: Lu, Chih-Yuan organization: Emerging Central Laboratory, Macronix International Co., Ltd., Hsinchu, Taiwan
BookMark	eNp9kM9LwzAUgINMcJveBS8Bz51J8_s45vzFnDJ2L2mazoy2qUl72H9vx4YHD54eD77v8fgmYNT4xgJwi9EMY6QetsvHWYqwmqVS0VSQCzDGjIlEccpHYIwQlokiklyBSYz7YeWUpmPw9mlD6UOtG2Pha91q00XoSzhvdOV3cGM383e49k3iCqsr1x2gb-Da9sHXPrRfzsCFr9u-c83uGlyWuor25jynYPu03C5ektXH8-tivkoMYaJLckOI1CznuhAF1lphW5RCWk2IFYgZw9Iy1zTXzBZYlIXKFS-QMFLmFHFMpuD-dLYN_ru3scv2vg_DuzFLsWSCYEr5QPETZYKPMdgyM67TnfNNF7SrMoyyY7ZsyJYds2XnbIOI_ohtcLUOh_-Uu5PirLW_uOQkVZSRH3wweks
CODEN	IETDAI
CitedBy_id	crossref_primary_10_1109_TED_2023_3308527 crossref_primary_10_1109_TED_2021_3118996 crossref_primary_10_1109_TCAD_2024_3514778 crossref_primary_10_1038_s41893_024_01352_4 crossref_primary_10_3390_electronics12153298 crossref_primary_10_1002_aisy_202200145 crossref_primary_10_3390_electronics10091063 crossref_primary_10_3389_fnins_2022_932270 crossref_primary_10_1038_s41598_021_00076_1 crossref_primary_10_1063_1_5124915 crossref_primary_10_1088_1361_6463_ac296d crossref_primary_10_1109_TED_2023_3244509 crossref_primary_10_1002_aisy_202300586 crossref_primary_10_1002_aisy_202300784 crossref_primary_10_1002_aelm_202300508 crossref_primary_10_1109_TED_2021_3068696 crossref_primary_10_1002_aisy_202000040 crossref_primary_10_1016_j_mejo_2024_106189 crossref_primary_10_1109_TVLSI_2024_3409648 crossref_primary_10_1109_TDMR_2022_3182133 crossref_primary_10_1109_TED_2021_3089995 crossref_primary_10_1088_1674_4926_42_1_013104 crossref_primary_10_1145_3759918 crossref_primary_10_1016_j_memori_2023_100053 crossref_primary_10_1109_TNANO_2020_3010070 crossref_primary_10_1002_pssa_202100753 crossref_primary_10_1109_LED_2021_3091995 crossref_primary_10_1109_TED_2019_2963323 crossref_primary_10_1109_TVLSI_2020_3020286 crossref_primary_10_1109_TED_2022_3186965 crossref_primary_10_1039_D3MH00508A crossref_primary_10_1088_2053_1583_acdfe1 crossref_primary_10_1109_TETC_2023_3237778 crossref_primary_10_3389_fnins_2021_749811
Cites_doi	10.1145/2749469.2750386 10.1109/IEDM.2017.8268369 10.1109/LED.2017.2732025 10.1109/IEDM.2012.6479105 10.1109/TED.2015.2439635 10.1109/TED.2014.2330200 10.1109/TED.2015.2477135 10.1038/nature14539 10.1109/IEDM.2015.7409622 10.1109/IEDM.2017.8268373 10.1109/TED.2015.2439812 10.1145/2897937.2898010 10.1109/IEDM.2017.8268522 10.1109/5.726791 10.1109/IEDM.2016.7838429 10.1109/IEDM.2017.8268313
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2019
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2019
DBID	97E RIA RIE AAYXX CITATION 7SP 8FD L7M
DOI	10.1109/TED.2019.2894273
DatabaseName	IEEE All-Society Periodicals Package (ASPP) 2005-present IEEE All-Society Periodicals Package (ASPP) 1998-Present IEEE Electronic Library (IEL) CrossRef Electronics & Communications Abstracts Technology Research Database Advanced Technologies Database with Aerospace
DatabaseTitle	CrossRef Technology Research Database Advanced Technologies Database with Aerospace Electronics & Communications Abstracts
DatabaseTitleList	Technology Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	1557-9646
EndPage	1295
ExternalDocumentID	10_1109_TED_2019_2894273 8632945
Genre	orig-research
GroupedDBID	-~X .DC 0R~ 29I 3EH 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK ACKIV ACNCT AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD F5P HZ~ H~9 IAAWW IBMZZ ICLAB IDIHD IFIPE IFJZH IPLJI JAVBF LAI M43 MS~ O9- OCL P2P RIA RIE RNS TAE TN5 VH1 VJK VOH AAYXX CITATION 7SP 8FD L7M RIG
ID	FETCH-LOGICAL-c357t-bc338a5b6ad7d1aa91edf78ea33e705cc52fba4ba5ed17fd9b96d07c88b40613
IEDL.DBID	RIE
ISICitedReferencesCount	65
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000460970400023&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0018-9383
IngestDate	Mon Jun 30 10:11:42 EDT 2025 Tue Nov 18 22:39:02 EST 2025 Sat Nov 29 03:02:19 EST 2025 Wed Aug 27 03:00:04 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	3
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c357t-bc338a5b6ad7d1aa91edf78ea33e705cc52fba4ba5ed17fd9b96d07c88b40613
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0003-1158-5289 0000-0001-8352-3584
PQID	2185731446
PQPubID	85466
PageCount	7
ParticipantIDs	crossref_citationtrail_10_1109_TED_2019_2894273 crossref_primary_10_1109_TED_2019_2894273 ieee_primary_8632945 proquest_journals_2185731446
PublicationCentury	2000
PublicationDate	2019-03-01
PublicationDateYYYYMMDD	2019-03-01
PublicationDate_xml	– month: 03 year: 2019 text: 2019-03-01 day: 01
PublicationDecade	2010
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	IEEE transactions on electron devices
PublicationTitleAbbrev	TED
PublicationYear	2019
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref15 ref11 ref10 ref17 piccolboni (ref19) 2014 ref18 chen (ref9) 2017 ref23 ref26 ref25 ref20 burr (ref7) 2016; 2 ref22 ref21 pedretti (ref2) 2017 chen (ref24) 2013 ref8 ref4 ho (ref14) 2016 ref3 ref6 hayakawa (ref13) 2015 ref5 kim (ref1) 2015 hsu (ref16) 2015 yan (ref12) 2017
References_xml	– ident: ref8 doi: 10.1145/2749469.2750386 – ident: ref3 doi: 10.1109/IEDM.2017.8268369 – volume: 2 start-page: 89 year: 2016 ident: ref7 article-title: Neuromorphic computing using non-volatile memory publication-title: Adv Phys X – ident: ref21 doi: 10.1109/LED.2017.2732025 – start-page: 11.4.1 year: 2017 ident: ref12 article-title: Understanding the trade-offs of device, circuit and application in ReRAM-based neuromorphic computing systems publication-title: IEDM Tech Dig – start-page: 17.1.1 year: 2015 ident: ref1 article-title: NVM neuromorphic core with 64k-cell (256-by-256) phase change memory synaptic array with on-chip neuron circuits for continuous in-situ learning publication-title: IEDM Tech Dig – start-page: 14t year: 2015 ident: ref13 article-title: Highly reliable TaOxReRAM with centralized filament for 28-nm embedded application publication-title: Proc VLSI Symp Tech – ident: ref25 doi: 10.1109/IEDM.2012.6479105 – start-page: 28 year: 2016 ident: ref14 article-title: Random soft error suppression by stoichiometric engineering: CMOS compatible and reliable 1Mb HfO2-ReRAM with 2 extra masks for embedded IoT systems publication-title: Proc VLSI Symp Tech – start-page: 1 year: 2014 ident: ref19 article-title: Investigation of HfO2/Ti based vertical RRAM-Performances and variability publication-title: Proc 14th Annu Non-Volatile Memory Technol Symp – ident: ref4 doi: 10.1109/TED.2015.2439635 – start-page: 1168 year: 2015 ident: ref16 article-title: A study of array resistance distribution and a novel operation algorithm for WOx ReRAM memory publication-title: Proc Int Conf SSDM – ident: ref18 doi: 10.1109/TED.2014.2330200 – start-page: 10.1.1 year: 2013 ident: ref24 article-title: Improvement of data retention in HfO2/Hf 1T1R RRAM cell under low operating current publication-title: IEDM Tech Dig – ident: ref20 doi: 10.1109/TED.2015.2477135 – ident: ref5 doi: 10.1038/nature14539 – start-page: 28.1.1 year: 2017 ident: ref2 article-title: Modeling-based design of brain-inspired spiking neural networks with RRAM learning synapses publication-title: IEDM Tech Dig – ident: ref6 doi: 10.1109/IEDM.2015.7409622 – ident: ref11 doi: 10.1109/IEDM.2017.8268373 – ident: ref22 doi: 10.1109/TED.2015.2439812 – ident: ref15 doi: 10.1145/2897937.2898010 – ident: ref26 doi: 10.1109/IEDM.2017.8268522 – ident: ref17 doi: 10.1109/5.726791 – ident: ref10 doi: 10.1109/IEDM.2016.7838429 – ident: ref23 doi: 10.1109/IEDM.2017.8268313 – start-page: 28.2.1 year: 2017 ident: ref9 article-title: A 16Mb dual-mode ReRAM macro with sub-14ns computing-in-memory and memory functions enabled by self-write termination scheme publication-title: IEDM Tech Dig
SSID	ssj0016442
Score	2.5147767
Snippet	Resistive random access memory (ReRAM) is often considered as a strong candidate for storing the weights in non-von Neumann neuromorphic computing systems....
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	1289
SubjectTerms	Accuracy Analog memory Artificial neural networks Computation Computer simulation Error reduction Inference Mapping Memory management Multilayers Neural networks Neuromorphic computing Neuromorphics noise Random access memory reliability Resistance resistive random access memory (ReRAM) stability Synapses Training Variation Weight
Title	Performance Impacts of Analog ReRAM Non-ideality on Neuromorphic Computing
URI	https://ieeexplore.ieee.org/document/8632945 https://www.proquest.com/docview/2185731446
Volume	66
WOSCitedRecordID	wos000460970400023&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1557-9646 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0016442 issn: 0018-9383 databaseCode: RIE dateStart: 19630101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8NAEB7a4kEPvqpYrbIHL4Jpk2y2mz0WsahoKaWH3sI-paCJ9OHvd3eTVkURvCWwu4RvHzNfZucbgEttiY8iLAliwkNLUKgKuApNQBRPFRXYiNQnCj_S4TCdTtmoBtebXBittb98pjvu0cfyVSFX7ldZN-3hmCWkDnVKaZmrtYkYWLteKoNHdgNb2rUOSYasa48Ad4eLdSy5SGKKv5kgX1Plx0Hsrctg73_ftQ-7lReJ-uW0H0BN54ew80VbsAkPo8-UAHTvcyEXqDDIqZAUz2isx_0nNCzyYKa098VRkSMv1fFaWOxnEpUFH-xgRzAZ3E5u7oKqcEIgMaHLQEhLPDkRPa6oijhnkVaGpppjrGlIpCSxETwRnGgVUaOYYD0VUpmmwtv3Y2jkRa5PANm5M8wF66KQW8fKCGrfuHWqcBJrYngLumsoM1mJirvaFi-ZJxchyyz4mQM_q8BvwdWmx1spqPFH26YDe9OuwrkF7fVsZdWOW2SxE7XCjt2e_t7rDLbd2OX9sTY0lvOVPoct-b6cLeYXfjF9ABjwxvw
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1bS8MwFD7MKagP3sV5zYMvgnVt0yzN4xDHpnPI2MPeSq4y0Fbc9PebpN1UFMG3FpK2fGlyzpeT8x2Ac22JjyIsCWLCQ0tQqAq4Ck1AFE8VFdiI1CcK9-lgkI7H7KEGl4tcGK21P3ymr9ylj-WrQr65rbJm2sIxS8gSLJMkiaMyW2sRM7CWvdQGj-wUtsRrHpQMWdMuAu4UF7uy9CKJKf5mhHxVlR9Lsbcvnc3_fdkWbFR-JGqXA78NNZ3vwPoXdcFduH34TApAPZ8NOUWFQU6HpHhEQz1s36NBkQcTpb03joocebGO58KiP5GoLPlgH7YHo87N6LobVKUTAokJnQVCWurJiWhxRVXEOYu0MjTVHGNNQyIliY3gieBEq4gaxQRrqZDKNBXewu9DPS9yfQDIjp5hLlwXhdy6VkZQe8etW4WTWBPDG9CcQ5nJSlbcVbd4yjy9CFlmwc8c-FkFfgMuFj1eSkmNP9ruOrAX7SqcG3A8H62smnPTLHayVtjx28Pfe53Band038_6vcHdEay595SnyY6hPnt90yewIt9nk-nrqf-xPgCeMcpD
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Performance+Impacts+of+Analog+ReRAM+Non-ideality+on+Neuromorphic+Computing&rft.jtitle=IEEE+transactions+on+electron+devices&rft.au=Yu-Hsuan%2C+Lin&rft.au=Chao-Hung%2C+Wang&rft.au=Lee%2C+Ming-Hsiu&rft.au=Dai-Ying%2C+Lee&rft.date=2019-03-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.issn=0018-9383&rft.eissn=1557-9646&rft.volume=66&rft.issue=3&rft.spage=1289&rft_id=info:doi/10.1109%2FTED.2019.2894273&rft.externalDBID=NO_FULL_TEXT
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9383&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9383&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9383&client=summon