Asynchrounous Decentralized Learning of a Neural Network

In this work, we exploit an asynchronous computing framework namely ARock to learn a deep neural network called self-size estimating feedforward neural network (SSFN) in a decentralized scenario. Using this algorithm namely asynchronous decentralized SSFN (dSSFN), we provide the centralized equivale...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings of the ... IEEE International Conference on Acoustics, Speech and Signal Processing (1998) s. 3947 - 3951
Hlavní autoři: Liang, Xinyue, Javid, Alireza M., Skoglund, Mikael, Chatterjee, Saikat
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 01.05.2020
Témata:
Asynchronous
Communication networks
convex optimization
decentralized learning
Feedforward neural networks
neural networks
Optimization
Signal processing
Signal processing algorithms
Sparse matrices
Speech processing
ISSN:2379-190X
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:In this work, we exploit an asynchronous computing framework namely ARock to learn a deep neural network called self-size estimating feedforward neural network (SSFN) in a decentralized scenario. Using this algorithm namely asynchronous decentralized SSFN (dSSFN), we provide the centralized equivalent solution under certain technical assumptions. Asynchronous dSSFN relaxes the communication bottleneck by allowing one node activation and one side communication, which reduces the communication overhead significantly, consequently increasing the learning speed. We compare asynchronous dSSFN with traditional synchronous dSSFN in the experimental results, which shows the competitive performance of asynchronous dSSFN, especially when the communication network is sparse.
ISSN:2379-190X
DOI:10.1109/ICASSP40776.2020.9053996