Parallel Selective Algorithms for Nonconvex Big Data Optimization

We propose a decomposition framework for the parallel optimization of the sum of a differentiable (possibly nonconvex) function and a (block) separable nonsmooth, convex one. The latter term is usually employed to enforce structure in the solution, typically sparsity. Our framework is very flexible...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on signal processing Vol. 63; no. 7; pp. 1874 - 1889
Main Authors: Facchinei, Francisco, Scutari, Gesualdo, Sagratella, Simone
Format: Journal Article
Language:English
Published: IEEE 01.04.2015
Subjects:
Approximation methods
Convergence
distributed methods
Jacobi method
Jacobian matrices
LASSO
Optimization
Parallel optimization
Signal processing algorithms
sparse solution
variables selection
Vectors
ISSN:1053-587X, 1941-0476
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We propose a decomposition framework for the parallel optimization of the sum of a differentiable (possibly nonconvex) function and a (block) separable nonsmooth, convex one. The latter term is usually employed to enforce structure in the solution, typically sparsity. Our framework is very flexible and includes both fully parallel Jacobi schemes and Gauss-Seidel (i.e., sequential) ones, as well as virtually all possibilities "in between" with only a subset of variables updated at each iteration. Our theoretical convergence results improve on existing ones, and numerical results on LASSO, logistic regression, and some nonconvex quadratic problems show that the new method consistently outperforms existing algorithms.
ISSN:1053-587X
1941-0476
DOI:10.1109/TSP.2015.2399858