Easily Parallelizable and Distributable Class of Algorithms for Structured Sparsity, with Optimal Acceleration

Many statistical learning problems can be posed as minimization of a sum of two convex functions, one typically a composition of nonsmooth and linear functions. Examples include regression under structured sparsity assumptions. Popular algorithms for solving such problems, for example, ADMM, often i...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Journal of computational and graphical statistics Ročník 28; číslo 4; s. 821 - 833
Hlavní autoři: Ko, Seyoon, Yu, Donghyeon, Won, Joong-Ho
Médium: Journal Article
Jazyk:angličtina
Vydáno: Alexandria Taylor & Francis 02.10.2019
American Statistical Association, the Institute of Mathematical Statistics, and the Interface Foundation of North America
Taylor & Francis Ltd
Témata:
Acceleration
Advances in Sampling and Optimization
Algorithms
Computer networks
Convergence
Distributed computing
Distributed processing
GPU
Linear functions
Machine learning
Monotone operator theory
Nonsmooth optimization
Operator splitting
Optimization
Parallel processing
Regression analysis
Sparsity
Statistical analysis
ISSN:1061-8600, 1537-2715
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Many statistical learning problems can be posed as minimization of a sum of two convex functions, one typically a composition of nonsmooth and linear functions. Examples include regression under structured sparsity assumptions. Popular algorithms for solving such problems, for example, ADMM, often involve nontrivial optimization subproblems or smoothing approximation. We consider two classes of primal-dual algorithms that do not incur these difficulties, and unify them from a perspective of monotone operator theory. From this unification, we propose a continuum of preconditioned forward-backward operator splitting algorithms amenable to parallel and distributed computing. For the entire region of convergence of the whole continuum of algorithms, we establish its rates of convergence. For some known instances of this continuum, our analysis closes the gap in theory. We further exploit the unification to propose a continuum of accelerated algorithms. We show that the whole continuum attains the theoretically optimal rate of convergence. The scalability of the proposed algorithms, as well as their convergence behavior, is demonstrated up to 1.2 million variables with a distributed implementation. The code is available at https://github.com/kose-y/dist-primal-dual . Supplemental materials for this article are available online.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1061-8600
1537-2715
DOI:10.1080/10618600.2019.1592757