Fast Distributed Gradient Methods

We study distributed optimization problems when N nodes minimize the sum of their individual costs subject to a common vector variable. The costs are convex, have Lipschitz continuous gradient (with constant L), and bounded gradient. We propose two fast distributed gradient algorithms based on the c...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transactions on automatic control Ročník 59; číslo 5; s. 1131 - 1146
Hlavní autoři: Jakovetic, Dusan, Xavier, Joao, Moura, Jose M. F.
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York IEEE 01.05.2014
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Acceleration
Algorithm design and analysis
Algorithms
Consensus
Constants
Convergence
convergence rate
Costs
distributed optimization
Educational institutions
Errors
Gradient methods
Mathematical analysis
Nesterov gradient
Optimization
Syntax
Vectors
ISSN:0018-9286, 1558-2523
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:We study distributed optimization problems when N nodes minimize the sum of their individual costs subject to a common vector variable. The costs are convex, have Lipschitz continuous gradient (with constant L), and bounded gradient. We propose two fast distributed gradient algorithms based on the centralized Nesterov gradient algorithm and establish their convergence rates in terms of the per-node communications K and the per-node gradient evaluations k. Our first method, Distributed Nesterov Gradient, achieves rates O( logK/K) and O(logk/k). Our second method, Distributed Nesterov gradient with Consensus iterations, assumes at all nodes knowledge of L and μ(W) - the second largest singular value of the N ×N doubly stochastic weight matrix W. It achieves rates O( 1/ K 2-ξ ) and O( 1/k 2 ) ( ξ > 0 arbitrarily small). Further, we give for both methods explicit dependence of the convergence constants on N and W. Simulation examples illustrate our findings.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:0018-9286
1558-2523
DOI:10.1109/TAC.2014.2298712