Synaptic metaplasticity in binarized neural networks
Unlike the brain, artificial neural networks, including state-of-the-art deep neural networks for computer vision, are subject to "catastrophic forgetting": they rapidly forget the previous task when trained on a new one. Neuroscience suggests that biological synapses avoid this issue thro...
Gespeichert in:
| Veröffentlicht in: | arXiv.org |
|---|---|
| Hauptverfasser: | , , , |
| Format: | Paper |
| Sprache: | Englisch |
| Veröffentlicht: |
Ithaca
Cornell University Library, arXiv.org
19.01.2021
|
| Schlagworte: | |
| ISSN: | 2331-8422 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | Unlike the brain, artificial neural networks, including state-of-the-art deep neural networks for computer vision, are subject to "catastrophic forgetting": they rapidly forget the previous task when trained on a new one. Neuroscience suggests that biological synapses avoid this issue through the process of synaptic consolidation and metaplasticity: the plasticity itself changes upon repeated synaptic events. In this work, we show that this concept of metaplasticity can be transferred to a particular type of deep neural networks, binarized neural networks, to reduce catastrophic forgetting. |
|---|---|
| AbstractList | Unlike the brain, artificial neural networks, including state-of-the-art deep neural networks for computer vision, are subject to "catastrophic forgetting": they rapidly forget the previous task when trained on a new one. Neuroscience suggests that biological synapses avoid this issue through the process of synaptic consolidation and metaplasticity: the plasticity itself changes upon repeated synaptic events. In this work, we show that this concept of metaplasticity can be transferred to a particular type of deep neural networks, binarized neural networks, to reduce catastrophic forgetting. |
| Author | Ernoult, Maxence Querlioz, Damien Laborieux, Axel Hirtzlin, Tifenn |
| Author_xml | – sequence: 1 givenname: Axel surname: Laborieux fullname: Laborieux, Axel – sequence: 2 givenname: Maxence surname: Ernoult fullname: Ernoult, Maxence – sequence: 3 givenname: Tifenn surname: Hirtzlin fullname: Hirtzlin, Tifenn – sequence: 4 givenname: Damien surname: Querlioz fullname: Querlioz, Damien |
| BookMark | eNotjU9LwzAcQIM4cG77AN4KnluTX_41RxnqhIEHdx-_NClk1rQmrTo_vQU9vXd675pcxj56Qm4YrUQtJb3D9B0-K2CUVVRLAxdkCZyzshYAV2ST84lSCkqDlHxJxOs54jCGpnj3Iw4d5tnDeC5CLGyImMKPd0X0U8JuxvjVp7e8JosWu-w3_1yRw-PDYbsr9y9Pz9v7fYkSTOkNaNvWziiusG59q62zFhkqq7BRjWmYtSCF5UYLaJ0T3DoqgatGOFDAV-T2Lzuk_mPyeTye-inF-XgEoQ2rNaWG_wL0MUmq |
| ContentType | Paper |
| Copyright | 2021. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
| Copyright_xml | – notice: 2021. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
| DBID | 8FE 8FG ABJCF ABUWG AFKRA AZQEC BENPR BGLVJ CCPQU DWQXO HCIFZ L6V M7S PHGZM PHGZT PIMPY PKEHL PQEST PQGLB PQQKQ PQUKI PRINS PTHSS |
| DOI | 10.48550/arxiv.2101.07592 |
| DatabaseName | ProQuest SciTech Collection ProQuest Technology Collection Materials Science & Engineering Collection ProQuest Central (Alumni Edition) ProQuest Central UK/Ireland ProQuest Central Essentials ProQuest Central ProQuest Technology Collection ProQuest One Community College ProQuest Central Korea SciTech Premium Collection ProQuest Engineering Collection Engineering Database ProQuest Central Premium ProQuest One Academic (New) Publicly Available Content Database ProQuest One Academic Middle East (New) ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Applied & Life Sciences ProQuest One Academic (retired) ProQuest One Academic UKI Edition ProQuest Central China Engineering Collection |
| DatabaseTitle | Publicly Available Content Database Engineering Database Technology Collection ProQuest One Academic Middle East (New) ProQuest Central Essentials ProQuest One Academic Eastern Edition ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest Technology Collection ProQuest SciTech Collection ProQuest Central China ProQuest Central ProQuest One Applied & Life Sciences ProQuest Engineering Collection ProQuest One Academic UKI Edition ProQuest Central Korea Materials Science & Engineering Collection ProQuest Central (New) ProQuest One Academic ProQuest One Academic (New) Engineering Collection |
| DatabaseTitleList | Publicly Available Content Database |
| Database_xml | – sequence: 1 dbid: PIMPY name: Publicly Available Content Database url: http://search.proquest.com/publiccontent sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Physics |
| EISSN | 2331-8422 |
| Genre | Working Paper/Pre-Print |
| GroupedDBID | 8FE 8FG ABJCF ABUWG AFKRA ALMA_UNASSIGNED_HOLDINGS AZQEC BENPR BGLVJ CCPQU DWQXO FRJ HCIFZ L6V M7S M~E PHGZM PHGZT PIMPY PKEHL PQEST PQGLB PQQKQ PQUKI PRINS PTHSS |
| ID | FETCH-LOGICAL-a529-e927bf8d9636a8fef7bdbba1a6b6ac6c9c1bb254b39742fdd43bd05236c4d2623 |
| IEDL.DBID | BENPR |
| IngestDate | Mon Jun 30 09:23:02 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a529-e927bf8d9636a8fef7bdbba1a6b6ac6c9c1bb254b39742fdd43bd05236c4d2623 |
| Notes | SourceType-Working Papers-1 ObjectType-Working Paper/Pre-Print-1 content type line 50 |
| OpenAccessLink | https://www.proquest.com/docview/2479187009?pq-origsite=%requestingapplication% |
| PQID | 2479187009 |
| PQPubID | 2050157 |
| ParticipantIDs | proquest_journals_2479187009 |
| PublicationCentury | 2000 |
| PublicationDate | 20210119 |
| PublicationDateYYYYMMDD | 2021-01-19 |
| PublicationDate_xml | – month: 01 year: 2021 text: 20210119 day: 19 |
| PublicationDecade | 2020 |
| PublicationPlace | Ithaca |
| PublicationPlace_xml | – name: Ithaca |
| PublicationTitle | arXiv.org |
| PublicationYear | 2021 |
| Publisher | Cornell University Library, arXiv.org |
| Publisher_xml | – name: Cornell University Library, arXiv.org |
| SSID | ssj0002672553 |
| Score | 1.7467358 |
| SecondaryResourceType | preprint |
| Snippet | Unlike the brain, artificial neural networks, including state-of-the-art deep neural networks for computer vision, are subject to "catastrophic forgetting":... |
| SourceID | proquest |
| SourceType | Aggregation Database |
| SubjectTerms | Artificial neural networks Computer vision Neural networks Synapses |
| Title | Synaptic metaplasticity in binarized neural networks |
| URI | https://www.proquest.com/docview/2479187009 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV07T8MwELagBYmJt3iUKgNr2thx_JiQQK1goIpohzJVfkqRIC1JqCi_HjtNQSwsTJblxTqfz9_dfb4D4Jpoq7kVvsIlYSE2EQtFokUYa-vgPKMmidfNJuhoxKZTnjYBt7KhVW5sYm2o9Vz5GHkfYcqhU66I3yzeQt81ymdXmxYa26DtK5XhFmjfDkbp03eUBRHqMHO8TmfWxbv6ovjIlj3n6cCeey7rFOhvI1y_LMP9_-7pALRTsTDFIdgy-RHYrRmdqjwGeLzKhbMHKng1lVg4kOz509UqyPJA-k-42afRgS9nKV7cUJPByxMwGQ4md_dh0yLBSRTx0HBEpWXa3SIimDWWSi2lgIJIIhRRXEEpnQsoHerAyGqNY6l9IJgorJFDPqeglc9zcwaCSDIYSSUUSwSmGsokssRdf4MsFyqS56CzkcGsUfNy9iOAi7-XL8Ee8mSQCIaQd0CrKt7NFdhRyyori25zal1PvBy7WfrwmD5_AZzRpp8 |
| linkProvider | ProQuest |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V07T8MwELaqFgQTb_EokAHGtInjOvaAGICKqg9VokOZKj-lSJCWphTKf-I_ck5bEAtbB6YMWZzzl-98d5_vELqg2mpuhetwSZlPTMB8UdPCj7SF4zyLTS2aD5uIOx3W7_NuAX0u78I4WeWSE3Oi1kPlcuRVTGIeArgCfj168d3UKFddXY7QmMOiaWZvELJlV41b2N9LjOt3vZt7fzFVABaBuW84jqVlGoBHBbPGxlJLKUJBJRWKKq5CKSFqkuCoCbZak0hqlzulimhMXZ8DYPwSAayzIip1G-3u43dSB9MYjujRvHqa9wqrivF7Mq1AYBVWwDvnFdffnJ87svrWPzPBNny6GJnxDiqYdBet53pVle0h8jBLBbCd8p7NRIwgBHDq8MnMS1JPuivGyYfRnmvWKZ7gkUvds33UW8U6D1AxHabmEHmBZGEglVCsJkisQ1kLLAVyM9hyoQJ5hMpLkw8WP3E2-LH38d-vz9HGfa_dGrQaneYJ2sRO9hKEfsjLqDgZv5pTtKamkyQbny0A46HBivfnC2N3AkQ |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Synaptic+metaplasticity+in+binarized+neural+networks&rft.jtitle=arXiv.org&rft.au=Laborieux%2C+Axel&rft.au=Ernoult%2C+Maxence&rft.au=Hirtzlin%2C+Tifenn&rft.au=Querlioz%2C+Damien&rft.date=2021-01-19&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422&rft_id=info:doi/10.48550%2Farxiv.2101.07592 |