Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE)

A growing body of work underlines striking similarities between biological neural networks and recurrent, binary neural networks. A relatively smaller body of work, however, addresses the similarities between learning dynamics employed in deep artificial neural networks and synaptic plasticity in sp...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Frontiers in neuroscience Jg. 14; S. 424
Hauptverfasser: Kaiser, Jacques, Mostafa, Hesham, Neftci, Emre
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Switzerland Frontiers Research Foundation 12.05.2020
Frontiers Media S.A
Schlagworte:
Back propagation
backpropagataon
Computers
Deep learning
embedded learning
Firing pattern
Learning algorithms
Machine learning
Neural networks
neuromorphic hardware
Neurons
Neuroplasticity
Neuroscience
Propagation
spiking neural network
surrogate gradient algorithm
Synaptic plasticity
neuromorphic hardware
surrogate gradient algorithm
embedded learning
spiking neural network
backpropagataon
ISSN:1662-453X, 1662-4548, 1662-453X
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:A growing body of work underlines striking similarities between biological neural networks and recurrent, binary neural networks. A relatively smaller body of work, however, addresses the similarities between learning dynamics employed in deep artificial neural networks and synaptic plasticity in spiking neural networks. The challenge preventing this is largely caused by the discrepancy between the dynamical properties of synaptic plasticity and the requirements for gradient backpropagation. Learning algorithms that approximate gradient backpropagation using local error functions can overcome this challenge. Here, we introduce Deep Continuous Local Learning (DECOLLE), a spiking neural network equipped with local error functions for online learning with no memory overhead for computing gradients. DECOLLE is capable of learning deep spatio temporal representations from spikes relying solely on local information, making it compatible with neurobiology and neuromorphic hardware. Synaptic plasticity rules are derived systematically from user-defined cost functions and neural dynamics by leveraging existing autodifferentiation methods of machine learning frameworks. We benchmark our approach on the event-based neuromorphic dataset N-MNIST and DvsGesture, on which DECOLLE performs comparably to the state-of-the-art. DECOLLE networks provide continuously learning machines that are relevant to biology and supportive of event-based, low-power computer vision architectures matching the accuracies of conventional computers on tasks where temporal precision and speed are essential.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
This article was submitted to Neuromorphic Engineering, a section of the journal Frontiers in Neuroscience
Edited by: Kaushik Roy, Purdue University, United States
Reviewed by: James Courtney Knight, University of Sussex, United Kingdom; Yulia Sandamirskaya, Intel, Germany
ISSN:1662-453X
1662-4548
1662-453X
DOI:10.3389/fnins.2020.00424