Parallel Multi-Block ADMM with o(1 / k) Convergence
This paper introduces a parallel and distributed algorithm for solving the following minimization problem with linear constraints: minimize f 1 ( x 1 ) + ⋯ + f N ( x N ) subject to A 1 x 1 + ⋯ + A N x N = c , x 1 ∈ X 1 , … , x N ∈ X N , where N ≥ 2 , f i are convex functions, A i are matrices, and X...
Uloženo v:
| Vydáno v: | Journal of scientific computing Ročník 71; číslo 2; s. 712 - 736 |
|---|---|
| Hlavní autoři: | , , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
New York
Springer US
01.05.2017
Springer Nature B.V |
| Témata: | |
| ISSN: | 0885-7474, 1573-7691 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Shrnutí: | This paper introduces a parallel and distributed algorithm for solving the following minimization problem with linear constraints:
minimize
f
1
(
x
1
)
+
⋯
+
f
N
(
x
N
)
subject to
A
1
x
1
+
⋯
+
A
N
x
N
=
c
,
x
1
∈
X
1
,
…
,
x
N
∈
X
N
,
where
N
≥
2
,
f
i
are convex functions,
A
i
are matrices, and
X
i
are feasible sets for variable
x
i
. Our algorithm extends the alternating direction method of multipliers (ADMM) and decomposes the original problem into
N
smaller subproblems and solves them in parallel at each iteration. This paper shows that the classic ADMM can be extended to the
N
-block Jacobi fashion and preserve convergence in the following two cases: (i) matrices
A
i
are mutually near-orthogonal and have full column-rank,
or
(ii) proximal terms are added to the
N
subproblems (but without any assumption on matrices
A
i
). In the latter case, certain proximal terms can let the subproblem be solved in more flexible and efficient ways. We show that
‖
x
k
+
1
-
x
k
‖
M
2
converges at a rate of
o
(1 /
k
) where
M
is a symmetric positive semi-definte matrix. Since the parameters used in the convergence analysis are conservative, we introduce a strategy for automatically tuning the parameters to substantially accelerate our algorithm in practice. We implemented our algorithm (for the case ii above) on Amazon EC2 and tested it on basis pursuit problems with >300 GB of distributed data. This is the first time that successfully solving a compressive sensing problem of such a large scale is reported. |
|---|---|
| Bibliografie: | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ISSN: | 0885-7474 1573-7691 |
| DOI: | 10.1007/s10915-016-0318-2 |