Optimal gradient tracking for decentralized optimization
In this paper, we focus on solving the decentralized optimization problem of minimizing the sum of n objective functions over a multi-agent network. The agents are embedded in an undirected graph where they can only send/receive information directly to/from their immediate neighbors. Assuming smooth...
Uloženo v:
| Vydáno v: | Mathematical programming Ročník 207; číslo 1-2; s. 1 - 53 |
|---|---|
| Hlavní autoři: | , , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
Berlin/Heidelberg
Springer Berlin Heidelberg
01.09.2024
|
| Témata: | |
| ISSN: | 0025-5610, 1436-4646 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Shrnutí: | In this paper, we focus on solving the decentralized optimization problem of minimizing the sum of
n
objective functions over a multi-agent network. The agents are embedded in an undirected graph where they can only send/receive information directly to/from their immediate neighbors. Assuming smooth and strongly convex objective functions, we propose an
Optimal Gradient Tracking
(
OGT
) method that achieves the optimal gradient computation complexity
O
κ
log
1
ϵ
and the optimal communication complexity
O
κ
θ
log
1
ϵ
simultaneously, where
κ
and
1
θ
denote the condition numbers related to the objective functions and the communication graph, respectively. To our best knowledge,
OGT
is the first single-loop decentralized gradient-type method that is optimal in both gradient computation and communication complexities. The development of
OGT
involves two building blocks that are also of independent interest. The first one is another new decentralized gradient tracking method termed
“Snapshot” Gradient Tracking
(
SS-GT
), which achieves the gradient computation and communication complexities of
O
κ
log
1
ϵ
and
O
κ
θ
log
1
ϵ
, respectively.
SS
-
G
T
can be potentially extended to more general settings compared to
OGT
. The second one is a technique termed
Loopless Chebyshev Acceleration
(LCA), which can be implemented “looplessly” but achieves a similar effect by adding multiple inner loops of Chebyshev acceleration in the algorithm. In addition to
SS
-
G
T
, this LCA technique can accelerate many other gradient tracking based methods with respect to the graph condition number
1
θ
. |
|---|---|
| ISSN: | 0025-5610 1436-4646 |
| DOI: | 10.1007/s10107-023-01997-7 |