The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware
The capabilities of natural neural systems have inspired new generations of machine learning algorithms as well as neuromorphic very large-scale integrated (VLSI) circuits capable of fast, low-power information processing. However, it has been argued that most modern machine learning algorithms are...
Uloženo v:
| Vydáno v: | arXiv.org |
|---|---|
| Hlavní autoři: | , , , , |
| Médium: | Paper |
| Jazyk: | angličtina |
| Vydáno: |
Ithaca
Cornell University Library, arXiv.org
26.08.2021
|
| Témata: | |
| ISSN: | 2331-8422 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | The capabilities of natural neural systems have inspired new generations of machine learning algorithms as well as neuromorphic very large-scale integrated (VLSI) circuits capable of fast, low-power information processing. However, it has been argued that most modern machine learning algorithms are not neurophysiologically plausible. In particular, the workhorse of modern deep learning, the backpropagation algorithm, has proven difficult to translate to neuromorphic hardware. In this study, we present a neuromorphic, spiking backpropagation algorithm based on synfire-gated dynamical information coordination and processing, implemented on Intel's Loihi neuromorphic research processor. We demonstrate a proof-of-principle three-layer circuit that learns to classify digits from the MNIST dataset. To our knowledge, this is the first work to show a Spiking Neural Network (SNN) implementation of the backpropagation algorithm that is fully on-chip, without a computer in the loop. It is competitive in accuracy with off-chip trained SNNs and achieves an energy-delay product suitable for edge computing. This implementation shows a path for using in-memory, massively parallel neuromorphic processors for low-power, low-latency implementation of modern deep learning applications. |
|---|---|
| AbstractList | The capabilities of natural neural systems have inspired new generations of machine learning algorithms as well as neuromorphic very large-scale integrated (VLSI) circuits capable of fast, low-power information processing. However, it has been argued that most modern machine learning algorithms are not neurophysiologically plausible. In particular, the workhorse of modern deep learning, the backpropagation algorithm, has proven difficult to translate to neuromorphic hardware. In this study, we present a neuromorphic, spiking backpropagation algorithm based on synfire-gated dynamical information coordination and processing, implemented on Intel's Loihi neuromorphic research processor. We demonstrate a proof-of-principle three-layer circuit that learns to classify digits from the MNIST dataset. To our knowledge, this is the first work to show a Spiking Neural Network (SNN) implementation of the backpropagation algorithm that is fully on-chip, without a computer in the loop. It is competitive in accuracy with off-chip trained SNNs and achieves an energy-delay product suitable for edge computing. This implementation shows a path for using in-memory, massively parallel neuromorphic processors for low-power, low-latency implementation of modern deep learning applications. |
| Author | Renner, Alpha Zlotnik, Anatoly Tao, Louis rest, Sheldon Sornborger, Andrew |
| Author_xml | – sequence: 1 givenname: Alpha surname: Renner fullname: Renner, Alpha – sequence: 2 givenname: Sheldon surname: rest fullname: rest, Sheldon – sequence: 3 givenname: Anatoly surname: Zlotnik fullname: Zlotnik, Anatoly – sequence: 4 givenname: Louis surname: Tao fullname: Tao, Louis – sequence: 5 givenname: Andrew surname: Sornborger fullname: Sornborger, Andrew |
| BookMark | eNotjstOwzAURC0EEqX0A9hZYp1wfR3nsSwV0KIKFmRfucl14jaJg5MAn08kWM1ijubMDbvsXEeM3QkIo1QpeND-x36FKCAOIQEJF2yBUoogjRCv2WoYTgCAcYJKyQV7zWvij7o49971utKjdR1fN5Xzdqxbvmv7hlrqRir5XHz09my7ir_R5F3rfF_bgm-1L7-1p1t2ZXQz0Oo_lyx_fso322D__rLbrPeBVigCERsBlGpJkaJCCTImi9XRYKYMEOiCMmPAoEipKE2JdMwyPUMyNpREEuWS3f_Nzo8_JxrGw8lNvpuNB1SREBgJFPIXXeRR3w |
| ContentType | Paper |
| Copyright | 2021. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
| Copyright_xml | – notice: 2021. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
| DBID | 8FE 8FG ABJCF ABUWG AFKRA AZQEC BENPR BGLVJ CCPQU DWQXO HCIFZ L6V M7S PHGZM PHGZT PIMPY PKEHL PQEST PQGLB PQQKQ PQUKI PRINS PTHSS |
| DOI | 10.48550/arxiv.2106.07030 |
| DatabaseName | ProQuest SciTech Collection ProQuest Technology Collection Materials Science & Engineering Collection ProQuest Central (Alumni) ProQuest Central UK/Ireland ProQuest Central Essentials ProQuest Central Technology collection ProQuest One Community College ProQuest Central SciTech Premium Collection ProQuest Engineering Collection Engineering Database ProQuest Databases ProQuest One Academic (New) Publicly Available Content Database ProQuest One Academic Middle East (New) ProQuest One Academic Eastern Edition (DO NOT USE) One Applied & Life Sciences ProQuest One Academic (retired) ProQuest One Academic UKI Edition ProQuest Central China Engineering collection |
| DatabaseTitle | Publicly Available Content Database Engineering Database Technology Collection ProQuest One Academic Middle East (New) ProQuest Central Essentials ProQuest One Academic Eastern Edition ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest Technology Collection ProQuest SciTech Collection ProQuest Central China ProQuest Central ProQuest One Applied & Life Sciences ProQuest Engineering Collection ProQuest One Academic UKI Edition ProQuest Central Korea Materials Science & Engineering Collection ProQuest Central (New) ProQuest One Academic ProQuest One Academic (New) Engineering Collection |
| DatabaseTitleList | Publicly Available Content Database |
| Database_xml | – sequence: 1 dbid: PIMPY name: Publicly Available Content Database url: http://search.proquest.com/publiccontent sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Physics |
| EISSN | 2331-8422 |
| Genre | Working Paper/Pre-Print |
| GroupedDBID | 8FE 8FG ABJCF ABUWG AFKRA ALMA_UNASSIGNED_HOLDINGS AZQEC BENPR BGLVJ CCPQU DWQXO FRJ HCIFZ L6V M7S M~E PHGZM PHGZT PIMPY PKEHL PQEST PQGLB PQQKQ PQUKI PRINS PTHSS |
| ID | FETCH-LOGICAL-a521-16f10e8a3e45ec51eff965bf295f0e0ace9ff0f218ecdfd2eb99aeff36fe74323 |
| IEDL.DBID | M7S |
| IngestDate | Mon Jun 30 09:27:02 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a521-16f10e8a3e45ec51eff965bf295f0e0ace9ff0f218ecdfd2eb99aeff36fe74323 |
| Notes | SourceType-Working Papers-1 ObjectType-Working Paper/Pre-Print-1 content type line 50 |
| OpenAccessLink | https://www.proquest.com/docview/2541124121?pq-origsite=%requestingapplication% |
| PQID | 2541124121 |
| PQPubID | 2050157 |
| ParticipantIDs | proquest_journals_2541124121 |
| PublicationCentury | 2000 |
| PublicationDate | 20210826 |
| PublicationDateYYYYMMDD | 2021-08-26 |
| PublicationDate_xml | – month: 08 year: 2021 text: 20210826 day: 26 |
| PublicationDecade | 2020 |
| PublicationPlace | Ithaca |
| PublicationPlace_xml | – name: Ithaca |
| PublicationTitle | arXiv.org |
| PublicationYear | 2021 |
| Publisher | Cornell University Library, arXiv.org |
| Publisher_xml | – name: Cornell University Library, arXiv.org |
| SSID | ssj0002672553 |
| Score | 1.7680235 |
| SecondaryResourceType | preprint |
| Snippet | The capabilities of natural neural systems have inspired new generations of machine learning algorithms as well as neuromorphic very large-scale integrated... |
| SourceID | proquest |
| SourceType | Aggregation Database |
| SubjectTerms | Algorithms Back propagation Data processing Deep learning Hardware Integrated circuits Machine learning Microprocessors Neuromorphic computing Power management Spiking Very large scale integration |
| Title | The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware |
| URI | https://www.proquest.com/docview/2541124121 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV07T8MwELagBYmJt3iUKgOraeI8PSGKWgESVUQ7lKmynTNEpQ-SUvj5nE0KEgMLY-TEss7Ofee7T98Rci7w1JjqGo1i5dMAhEsRRjjF0FUg4PPM18o2m4h7vWQ45GmVcCsrWuXKJ1pHnc2UyZG38CKDoUHgMe9y_kpN1yhTXa1aaKyTulFJ8Cx1r_-dY2FRjBGz_1XMtNJdLVF85MsLvOcYzU7Lff7lgi2udLf_u6IdUk_FHIpdsgbTPbJp-Zyq3Cd3uP1OW6gxzokuw5rfuXp5wu8XzxPHagJbOc7MwYH-PDcZc8cqdUxmaPpcOaam_y4KOCCDbmdwfUOrtglUIBZTL9KeC4nwIQhBhR5ozaNQasZD7YIrFHCtXY3QDirTGQPJucCX_EgDhhPMPyS16WwKR8ThrlJMgkhkEASZiiVnSspEhr6MPZ2Ex6SxssyoOvrl6McsJ38Pn5ItZggiLv6qUYPUFsUbnJENtVzkZdEk9Xanlz407Y7iU3p7nz5-Aqe5rXQ |
| linkProvider | ProQuest |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1LTwIxEG6IaPTkOz5Q96DHld3uswdjfBEQJCRw4Eba7lQ3ysMFQX-U_9FpAU08eOPguXvo7nz7zUxn-g0hpxxRo6trdhhJz_aBOza6EWZj6MrR4bPEU9IMm4jq9bjdZo0c-ZzfhdFtlXNONESd9KU-Iy9iIoOhge9S93LwauupUbq6Oh-hMYVFFT4mmLINLyq3aN8zSkt3rZuyPZsqYHN0VbYbKteBmHvgByADF5RiYSAUZYFywOESmFKOQs8HMlEJBcEYx4e8UAF6W61zgIyfxyiCMtMp2Pw-0qFhhAG6N62dGqWwIs_e0_E5plVaItS0Wv9ifOPGSuv_7ANskHyDDyDbJDnobZEV060qh9vkHsFtXXP5jK-AhGjAZV29POJ2R09dyygeG7HRxMKF5iDV9QDL6JB0-wisVFq6Y2HCM9ghrUXsfpcs9fo92CMWc6SkAngsfN9PZCQYlULEIvBE5Ko42CeFuSE6sx972PmxwsHfyydktdx6qHVqlXr1kKxR3QrjICmFBbI0yt7giCzL8SgdZscGRBbpLNhmX4qkCiU |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+Backpropagation+Algorithm+Implemented+on+Spiking+Neuromorphic+Hardware&rft.jtitle=arXiv.org&rft.au=Renner%2C+Alpha&rft.au=rest%2C+Sheldon&rft.au=Zlotnik%2C+Anatoly&rft.au=Tao%2C+Louis&rft.date=2021-08-26&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422&rft_id=info:doi/10.48550%2Farxiv.2106.07030 |