Synaptic metaplasticity in binarized neural networks

Unlike the brain, artificial neural networks, including state-of-the-art deep neural networks for computer vision, are subject to "catastrophic forgetting": they rapidly forget the previous task when trained on a new one. Neuroscience suggests that biological synapses avoid this issue thro...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	arXiv.org
Hlavní autoři:	Laborieux, Axel, Ernoult, Maxence, Hirtzlin, Tifenn, Querlioz, Damien
Médium:	Paper
Jazyk:	angličtina
Vydáno:	Ithaca Cornell University Library, arXiv.org 19.01.2021
Témata:	Artificial neural networks Computer vision Neural networks Synapses
ISSN:	2331-8422
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Popis
Shrnutí:	Unlike the brain, artificial neural networks, including state-of-the-art deep neural networks for computer vision, are subject to "catastrophic forgetting": they rapidly forget the previous task when trained on a new one. Neuroscience suggests that biological synapses avoid this issue through the process of synaptic consolidation and metaplasticity: the plasticity itself changes upon repeated synaptic events. In this work, we show that this concept of metaplasticity can be transferred to a particular type of deep neural networks, binarized neural networks, to reduce catastrophic forgetting.
Bibliografie:	SourceType-Working Papers-1 ObjectType-Working Paper/Pre-Print-1 content type line 50
ISSN:	2331-8422
DOI:	10.48550/arxiv.2101.07592