Gradual Surrogate Gradient Learning in Deep Spiking Neural Networks
Spiking Neural Network (SNN) is a promising solution for ultra-low-power hardware. Recent SNNs have reached the performance of Deep Neural Networks (DNNs) in dealing with many tasks. However, these methods often suffer from a long simulation time to achieve the accurate spike train information. In a...
Saved in:
| Published in: | Proceedings of the ... IEEE International Conference on Acoustics, Speech and Signal Processing (1998) pp. 8927 - 8931 |
|---|---|
| Main Authors: | , , , |
| Format: | Conference Proceeding |
| Language: | English |
| Published: |
IEEE
23.05.2022
|
| Subjects: | |
| ISSN: | 2379-190X |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | Spiking Neural Network (SNN) is a promising solution for ultra-low-power hardware. Recent SNNs have reached the performance of Deep Neural Networks (DNNs) in dealing with many tasks. However, these methods often suffer from a long simulation time to achieve the accurate spike train information. In addition, these methods are contingent on a well-designed initialization to effectively transmit the gradient information. To address these issues, we propose the Internal Spiking Neuron Model (ISNM), which uses the synaptic current instead of spike trains as the carrier of information. In addition, we design a gradual surrogate gradient learning algorithm to ensure that SNNs effectively back-propagate gradient information in the early stage of training and more accurate gradient information in the later stage of training. The experiments on various network structures on CIFAR-10 and CIFAR-100 datasets show that the proposed method can exceed the performance of previous SNN methods within 5 time steps. |
|---|---|
| AbstractList | Spiking Neural Network (SNN) is a promising solution for ultra-low-power hardware. Recent SNNs have reached the performance of Deep Neural Networks (DNNs) in dealing with many tasks. However, these methods often suffer from a long simulation time to achieve the accurate spike train information. In addition, these methods are contingent on a well-designed initialization to effectively transmit the gradient information. To address these issues, we propose the Internal Spiking Neuron Model (ISNM), which uses the synaptic current instead of spike trains as the carrier of information. In addition, we design a gradual surrogate gradient learning algorithm to ensure that SNNs effectively back-propagate gradient information in the early stage of training and more accurate gradient information in the later stage of training. The experiments on various network structures on CIFAR-10 and CIFAR-100 datasets show that the proposed method can exceed the performance of previous SNN methods within 5 time steps. |
| Author | Chen, Yi Zhang, Silin Qu, Hong Ren, Shiyu |
| Author_xml | – sequence: 1 givenname: Yi surname: Chen fullname: Chen, Yi organization: University of Electronic Science and Technology of China,School of Computer Science and Engineering,China – sequence: 2 givenname: Silin surname: Zhang fullname: Zhang, Silin organization: University of Electronic Science and Technology of China,School of Computer Science and Engineering,China – sequence: 3 givenname: Shiyu surname: Ren fullname: Ren, Shiyu organization: University of Electronic Science and Technology of China,School of Computer Science and Engineering,China – sequence: 4 givenname: Hong surname: Qu fullname: Qu, Hong email: hongqu@uestc.edu.cn organization: University of Electronic Science and Technology of China,School of Computer Science and Engineering,China |
| BookMark | eNotT11LxDAQjKLg9fQX-JI_0HObpPl4lKqnUE6hCr4dSbs94p1tSVvEf28OD5ZdZphZZhJy0fUdEkIzWGUZmLuX4r6q3gQ3jK0YxGWUkEqJM5JkUuYC4shzsmBcmTQz8HlFknH8AgCthF6QYh1sM9sDreYQ-p2dkB4Zj91ES7Sh892O-o4-IA60Gvz-iDc4h2jZ4PTTh_14TS5bexjx5nSX5OPp8b14TsvXdcxXpp4Bn1LOmrrV6IxxVqDLa2m00roGlTuBUpgcnMljKq4aF2uA1aqNYkAnWaMYX5Lb_78eEbdD8N82_G5PhfkfzK9NAw |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/ICASSP43922.2022.9746774 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISBN | 1665405406 9781665405409 |
| EISSN | 2379-190X |
| EndPage | 8931 |
| ExternalDocumentID | 9746774 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: China Postdoctoral Science Foundation funderid: 10.13039/501100002858 – fundername: National Science Foundation funderid: 10.13039/100000001 |
| GroupedDBID | 23M 6IE 6IF 6IH 6IK 6IL 6IM 6IN AAJGR AAWTH ABLEC ACGFS ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IJVOP IPLJI M43 OCL RIE RIL RIO RNS |
| ID | FETCH-LOGICAL-i203t-32dcf8eb99ba4eb5c698788c075b4e64950b9574837db9740a87f99b0eb62d723 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 10 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000864187909048&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:25:03 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i203t-32dcf8eb99ba4eb5c698788c075b4e64950b9574837db9740a87f99b0eb62d723 |
| PageCount | 5 |
| ParticipantIDs | ieee_primary_9746774 |
| PublicationCentury | 2000 |
| PublicationDate | 2022-May-23 |
| PublicationDateYYYYMMDD | 2022-05-23 |
| PublicationDate_xml | – month: 05 year: 2022 text: 2022-May-23 day: 23 |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings of the ... IEEE International Conference on Acoustics, Speech and Signal Processing (1998) |
| PublicationTitleAbbrev | ICASSP |
| PublicationYear | 2022 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0008748 |
| Score | 2.3183744 |
| Snippet | Spiking Neural Network (SNN) is a promising solution for ultra-low-power hardware. Recent SNNs have reached the performance of Deep Neural Networks (DNNs) in... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 8927 |
| SubjectTerms | Deep learning Hardware Learning systems Neurons Signal processing Signal processing algorithms Spiking Neural Networks Spiking Neuron Model Surrogate Gradient Training |
| Title | Gradual Surrogate Gradient Learning in Deep Spiking Neural Networks |
| URI | https://ieeexplore.ieee.org/document/9746774 |
| WOSCitedRecordID | wos000864187909048&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEB5q8aAXH634JgePrl03aR5HqVYFKYVV6K3sJFPZy7ZsW3-_ye5aFbx4CSEkJExIJpmZ7xuAK0ShOFe3kX8r8MifRBWhchg5qXGmtORYk7i-qNFITyZm3ILrDRaGiKrgM7oJ1cqX7-Z2HUxlPRNyYyixBVtKyRqrtbl1tRL6K1InNr3nwV2ajr22TQLayhfN2F9JVCodMtz73-z70P0G47HxRs0cQIuKQ9j9wSPYgcFjmQVMFUvXZTkPhjEWWgLUkTUEqu8sL9g90YKlizyYx1mg5fBDRnUc-LILb8OH18FT1GRHiPIk5quIJ87ONKExmAnCvpVG-_-s9W8AFCT9xydG01eBMd6hX3ucaTXznWPy4ncq4UfQLuYFHQNDm2mUsc3QkXAuMU5I189QklVCuOwEOkEc00VNgDFtJHH6d_MZ7ASJBxd7ws-hvSrXdAHb9mOVL8vLatc-AS6kmSQ |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NT8MgFH-Z00S9-DGN33LwaF0ttMDRTOcWZ7OkM9lt6QNmeumWbvPvF9o6NfHihRACgTwCD957v98DuEFknFJ-79m3AvXsSeQeco2ejgROuYgoViSuAx7HYjyWwwbcrrEwxpgy-MzcuWrpy9cztXKmsrZ0uTE424DNkLHAr9Ba63tXcCa-YnV82e53HpJkaPVt4PBWtqhH_0qjUmqR7t7_5t-Ho284HhmuFc0BNEx-CLs_mARb0HkuUoeqIsmqKGbONEZciwM7kppC9Z1kOXk0Zk6SeeYM5MQRc9ghcRUJvjiCt-7TqNPz6vwIXhb4dOnRQKupMCglpsxgqCIp7I9W2VcAMhPZr4-PMuSOM16jXbufCj61nX1jN0DzgB5DM5_l5gQIqlRg5KsUtWFaB1KzSIcpRkZxxnR6Ci0njsm8osCY1JI4-7v5GrZ7o9fBZNCPX85hx0nfOdwDegHNZbEyl7ClPpbZorgqd_ATVSScaw |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=proceeding&rft.title=Proceedings+of+the+...+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%281998%29&rft.atitle=Gradual+Surrogate+Gradient+Learning+in+Deep+Spiking+Neural+Networks&rft.au=Chen%2C+Yi&rft.au=Zhang%2C+Silin&rft.au=Ren%2C+Shiyu&rft.au=Qu%2C+Hong&rft.date=2022-05-23&rft.pub=IEEE&rft.eissn=2379-190X&rft.spage=8927&rft.epage=8931&rft_id=info:doi/10.1109%2FICASSP43922.2022.9746774&rft.externalDocID=9746774 |