Advances in Asynchronous Parallel and Distributed Optimization

Motivated by large-scale optimization problems arising in the context of machine learning, there have been several advances in the study of asynchronous parallel and distributed optimization methods during the past decade. Asynchronous methods do not require all processors to maintain a consistent v...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Proceedings of the IEEE Jg. 108; H. 11; S. 2013 - 2031
Hauptverfasser: Assran, By Mahmoud, Aytekin, Arda, Feyzmahdavian, Hamid Reza, Johansson, Mikael, Rabbat, Michael G.
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York IEEE 01.11.2020
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:
Computational efficiency
Computational modeling
Convergence
Design optimization
Distributed algorithms
Machine learning
machine learning algorithms
Microprocessors
Optimization
Optimization methods
parallel algorithms
Processors
Program processors
ISSN:0018-9219, 1558-2256, 1558-2256
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Motivated by large-scale optimization problems arising in the context of machine learning, there have been several advances in the study of asynchronous parallel and distributed optimization methods during the past decade. Asynchronous methods do not require all processors to maintain a consistent view of the optimization variables. Consequently, they generally can make more efficient use of computational resources than synchronous methods, and they are not sensitive to issues like stragglers (i.e., slow nodes) and unreliable communication links. Mathematical modeling of asynchronous methods involves proper accounting of information delays, which makes their analysis challenging. This article reviews recent developments in the design and analysis of asynchronous optimization methods, covering both centralized methods, where all processors update a master copy of the optimization variables, and decentralized methods, where each processor maintains a local copy of the variables. The analysis provides insights into how the degree of asynchrony impacts convergence rates, especially in stochastic optimization methods.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0018-9219
1558-2256
1558-2256
DOI:10.1109/JPROC.2020.3026619