The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware

The capabilities of natural neural systems have inspired new generations of machine learning algorithms as well as neuromorphic very large-scale integrated (VLSI) circuits capable of fast, low-power information processing. However, it has been argued that most modern machine learning algorithms are...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:arXiv.org
Hauptverfasser: Renner, Alpha, rest, Sheldon, Zlotnik, Anatoly, Tao, Louis, Sornborger, Andrew
Format: Paper
Sprache:Englisch
Veröffentlicht: Ithaca Cornell University Library, arXiv.org 26.08.2021
Schlagworte:
Algorithms
Back propagation
Data processing
Deep learning
Hardware
Integrated circuits
Machine learning
Microprocessors
Neuromorphic computing
Power management
Spiking
Very large scale integration
ISSN:2331-8422
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Abstract The capabilities of natural neural systems have inspired new generations of machine learning algorithms as well as neuromorphic very large-scale integrated (VLSI) circuits capable of fast, low-power information processing. However, it has been argued that most modern machine learning algorithms are not neurophysiologically plausible. In particular, the workhorse of modern deep learning, the backpropagation algorithm, has proven difficult to translate to neuromorphic hardware. In this study, we present a neuromorphic, spiking backpropagation algorithm based on synfire-gated dynamical information coordination and processing, implemented on Intel's Loihi neuromorphic research processor. We demonstrate a proof-of-principle three-layer circuit that learns to classify digits from the MNIST dataset. To our knowledge, this is the first work to show a Spiking Neural Network (SNN) implementation of the backpropagation algorithm that is fully on-chip, without a computer in the loop. It is competitive in accuracy with off-chip trained SNNs and achieves an energy-delay product suitable for edge computing. This implementation shows a path for using in-memory, massively parallel neuromorphic processors for low-power, low-latency implementation of modern deep learning applications.
AbstractList The capabilities of natural neural systems have inspired new generations of machine learning algorithms as well as neuromorphic very large-scale integrated (VLSI) circuits capable of fast, low-power information processing. However, it has been argued that most modern machine learning algorithms are not neurophysiologically plausible. In particular, the workhorse of modern deep learning, the backpropagation algorithm, has proven difficult to translate to neuromorphic hardware. In this study, we present a neuromorphic, spiking backpropagation algorithm based on synfire-gated dynamical information coordination and processing, implemented on Intel's Loihi neuromorphic research processor. We demonstrate a proof-of-principle three-layer circuit that learns to classify digits from the MNIST dataset. To our knowledge, this is the first work to show a Spiking Neural Network (SNN) implementation of the backpropagation algorithm that is fully on-chip, without a computer in the loop. It is competitive in accuracy with off-chip trained SNNs and achieves an energy-delay product suitable for edge computing. This implementation shows a path for using in-memory, massively parallel neuromorphic processors for low-power, low-latency implementation of modern deep learning applications.
Author Renner, Alpha
Zlotnik, Anatoly
Tao, Louis
rest, Sheldon
Sornborger, Andrew
Author_xml – sequence: 1
  givenname: Alpha
  surname: Renner
  fullname: Renner, Alpha
– sequence: 2
  givenname: Sheldon
  surname: rest
  fullname: rest, Sheldon
– sequence: 3
  givenname: Anatoly
  surname: Zlotnik
  fullname: Zlotnik, Anatoly
– sequence: 4
  givenname: Louis
  surname: Tao
  fullname: Tao, Louis
– sequence: 5
  givenname: Andrew
  surname: Sornborger
  fullname: Sornborger, Andrew
BookMark eNotjstOwzAURC0EEqX0A9hZYp1wfR3nsSwV0KIKFmRfucl14jaJg5MAn08kWM1ijubMDbvsXEeM3QkIo1QpeND-x36FKCAOIQEJF2yBUoogjRCv2WoYTgCAcYJKyQV7zWvij7o49971utKjdR1fN5Xzdqxbvmv7hlrqRir5XHz09my7ir_R5F3rfF_bgm-1L7-1p1t2ZXQz0Oo_lyx_fso322D__rLbrPeBVigCERsBlGpJkaJCCTImi9XRYKYMEOiCMmPAoEipKE2JdMwyPUMyNpREEuWS3f_Nzo8_JxrGw8lNvpuNB1SREBgJFPIXXeRR3w
ContentType Paper
Copyright 2021. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Copyright_xml – notice: 2021. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
DBID 8FE
8FG
ABJCF
ABUWG
AFKRA
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
HCIFZ
L6V
M7S
PHGZM
PHGZT
PIMPY
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
PTHSS
DOI 10.48550/arxiv.2106.07030
DatabaseName ProQuest SciTech Collection
ProQuest Technology Collection
Materials Science & Engineering Collection
ProQuest Central (Alumni)
ProQuest Central UK/Ireland
ProQuest Central Essentials - QC
ProQuest Central
ProQuest Technology Collection
ProQuest One Community College
ProQuest Central Korea
SciTech Premium Collection
ProQuest Engineering Collection
Engineering Database
ProQuest Central Premium
ProQuest One Academic (New)
ProQuest Publicly Available Content Database
ProQuest One Academic Middle East (New)
ProQuest One Academic Eastern Edition (DO NOT USE)
ProQuest One Applied & Life Sciences
ProQuest One Academic (retired)
ProQuest One Academic UKI Edition
ProQuest Central China
Engineering Collection
DatabaseTitle Publicly Available Content Database
Engineering Database
Technology Collection
ProQuest One Academic Middle East (New)
ProQuest Central Essentials
ProQuest One Academic Eastern Edition
ProQuest Central (Alumni Edition)
SciTech Premium Collection
ProQuest One Community College
ProQuest Technology Collection
ProQuest SciTech Collection
ProQuest Central China
ProQuest Central
ProQuest One Applied & Life Sciences
ProQuest Engineering Collection
ProQuest One Academic UKI Edition
ProQuest Central Korea
Materials Science & Engineering Collection
ProQuest Central (New)
ProQuest One Academic
ProQuest One Academic (New)
Engineering Collection
DatabaseTitleList Publicly Available Content Database
Database_xml – sequence: 1
  dbid: PIMPY
  name: ProQuest Publicly Available Content Database
  url: http://search.proquest.com/publiccontent
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Physics
EISSN 2331-8422
Genre Working Paper/Pre-Print
GroupedDBID 8FE
8FG
ABJCF
ABUWG
AFKRA
ALMA_UNASSIGNED_HOLDINGS
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
FRJ
HCIFZ
L6V
M7S
M~E
PHGZM
PHGZT
PIMPY
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
PTHSS
ID FETCH-LOGICAL-a521-16f10e8a3e45ec51eff965bf295f0e0ace9ff0f218ecdfd2eb99aeff36fe74323
IEDL.DBID M7S
IngestDate Mon Jun 30 09:27:02 EDT 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a521-16f10e8a3e45ec51eff965bf295f0e0ace9ff0f218ecdfd2eb99aeff36fe74323
Notes SourceType-Working Papers-1
ObjectType-Working Paper/Pre-Print-1
content type line 50
OpenAccessLink https://www.proquest.com/docview/2541124121?pq-origsite=%requestingapplication%
PQID 2541124121
PQPubID 2050157
ParticipantIDs proquest_journals_2541124121
PublicationCentury 2000
PublicationDate 20210826
PublicationDateYYYYMMDD 2021-08-26
PublicationDate_xml – month: 08
  year: 2021
  text: 20210826
  day: 26
PublicationDecade 2020
PublicationPlace Ithaca
PublicationPlace_xml – name: Ithaca
PublicationTitle arXiv.org
PublicationYear 2021
Publisher Cornell University Library, arXiv.org
Publisher_xml – name: Cornell University Library, arXiv.org
SSID ssj0002672553
Score 1.7681227
SecondaryResourceType preprint
Snippet The capabilities of natural neural systems have inspired new generations of machine learning algorithms as well as neuromorphic very large-scale integrated...
SourceID proquest
SourceType Aggregation Database
SubjectTerms Algorithms
Back propagation
Data processing
Deep learning
Hardware
Integrated circuits
Machine learning
Microprocessors
Neuromorphic computing
Power management
Spiking
Very large scale integration
Title The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware
URI https://www.proquest.com/docview/2541124121
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV07T8MwELagBYmJt3hWGVhDHSdx4wlR1AqQqCLaoUyV7Zwhgj5ISuHnczYpSAwsjJGTKDo7370-fUfImRRM2tTHhzCmfhRn3FfaUB84U0wkKkq4ccMmWr1eMhyKtCq4lRWtcomJDqizqbY18iYmMhgaRAELLmavvp0aZbur1QiNVVK3KgmBo-71v2ssjLcwYg6_mplOuqspi498cY55jtXsdNznXxDs_Ep3879ftEXqqZxBsU1WYLJD1h2fU5e75Ba332tL_YzvRMhw5vcuXx7x-fnT2HOawE6OM_NwoT_LbcXcc0od4ymaPtee7em_ywL2yKDbGVxd-9XYBF-iL_YDbgIKiQwhikHHARgjeKwME7GhQKUGYQw16NpBZyZjoISQeFPIDWA4wcJ9UptMJ3BAPM4RgowOlKAmElQrylQEEaIAuj3Fk0NysrTMqDr65ejHLEd_Lx-TDWYJIhR_VX5CavPiDU7Jml7M87JokHq700vvG25H8Sq9uUsfPgFVQqzl
linkProvider ProQuest
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V25TsQwELUQh6DiFjcuoAzrOImJC4S4VpwrpN1iu8h2xrCCPUiW66P4R8aGgERBR0HtFE5m8mbG8_yGkC0luXKlTwBRwoI4yUWgjWUBCK65THWcCuuHTew2Gmm7La9HyFt1F8bRKitM9ECd9407I69hIYOpQRzycH_wELipUa67Wo3Q-HCLC3h9xpKt3Ds7Rvtuc14_aR2dBp9TBQKFoSoIhQ0ZpCqCOAGThGCtFIm2XCaWAVMGpLXMYuQDk9ucg5ZS4UORsIDR1ukcIOKPYRbBpWcKNr-OdLjYxQQ9-uideqWwmipeOk87WFY5iVBPtf6B-D6M1af_2QeYIWPXagDFLBmB3hyZ8GxVU86Tc3RueqjMHb4CAqJ3Lnpwf4PbHd52qVc89mKjOcWF5qDj-gHU65B0--hYHUMdY-FZFbBAWn-x-0Uy2uv3YIlQIRBgrQm1ZDaWzGjGdQwxYhwGdS3SZbJWGSL7_LHL7NsKK78vb5LJ09bVZXZ51rhYJVPcUWEYgpJYI6PD4hHWybh5GnbKYsM7ESXZH9vsHXp5CZY
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+Backpropagation+Algorithm+Implemented+on+Spiking+Neuromorphic+Hardware&rft.jtitle=arXiv.org&rft.au=Renner%2C+Alpha&rft.au=rest%2C+Sheldon&rft.au=Zlotnik%2C+Anatoly&rft.au=Tao%2C+Louis&rft.date=2021-08-26&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422&rft_id=info:doi/10.48550%2Farxiv.2106.07030