Backpropagation Algorithms for a Broad Class of Dynamic Networks

This paper introduces a general framework for describing dynamic neural networks-the layered digital dynamic network (LDDN). This framework allows the development of two general algorithms for computing the gradients and Jacobians for these dynamic networks: backpropagation-through-time (BPTT) and r...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transactions on neural networks Ročník 18; číslo 1; s. 14 - 27
Hlavní autoři: Orlando De Jesus, Hagan, M.T.
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York, NY IEEE 01.01.2007
Institute of Electrical and Electronics Engineers
Témata:
Algorithms
Applied sciences
Artificial Intelligence
Backpropagation algorithms
Backpropagation through time (BPTT)
Cluster Analysis
Computer networks
Computer science; control theory; systems
Computer systems and distributed systems. User interface
Computing Methodologies
Connectionism. Neural networks
Delay lines
Digital
dynamic neural networks
Dynamics
Exact sciences and technology
gradient
Heuristic algorithms
Jacobian
Jacobian matrices
Jacobians
layered digital dynamic network (LDDN)
Learning
Mathematical analysis
Networks
Neural networks
Neural Networks (Computer)
Neurofeedback
Output feedback
Pattern Recognition, Automated - methods
Power engineering and energy
real-time recurrent learning (RTRL)
Recurrent neural networks
Signal Processing, Computer-Assisted
Software
Backpropagation
Recurrence
Recurrent neural nets
Gradient
dynamic neural networks
Neural network
Distributed system
Jacobi matrix
Real time
Search algorithm
Backpropagation algorithm
Backpropagation through time (BPTT)
real-time recurrent learning (RTRL)
recurrent neural networks
Jacobian
layered digital dynamic network (LDDN)
ISSN:1045-9227
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:This paper introduces a general framework for describing dynamic neural networks-the layered digital dynamic network (LDDN). This framework allows the development of two general algorithms for computing the gradients and Jacobians for these dynamic networks: backpropagation-through-time (BPTT) and real-time recurrent learning (RTRL). The structure of the LDDN framework enables an efficient implementation of both algorithms for arbitrary dynamic networks. This paper demonstrates that the BPTT algorithm is more efficient for gradient calculations, but the RTRL algorithm is more efficient for Jacobian calculations
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:1045-9227
DOI:10.1109/TNN.2006.882371