Exterior-Point Optimization for Sparse and Low-Rank Optimization

Many problems of substantial current interest in machine learning, statistics, and data science can be formulated as sparse and low-rank optimization problems. In this paper, we present the nonconvex exterior-point optimization solver (NExOS)—a first-order algorithm tailored to sparse and low-rank o...

Full description

Saved in:
Bibliographic Details
Published in:Journal of optimization theory and applications Vol. 202; no. 2; pp. 795 - 833
Main Authors: Das Gupta, Shuvomoy, Stellato, Bartolomeo, Van Parys, Bart P. G.
Format: Journal Article
Language:English
Published: New York Springer US 01.08.2024
Springer Nature B.V
Subjects:
Algorithms
Applications of Mathematics
Calculus of Variations and Optimal Control; Optimization
Constraints
Convexity
Engineering
First order algorithms
Machine learning
Mathematics
Mathematics and Statistics
Operations Research/Decision Theory
Optimization
Parameters
Theory of Computation
Sparse optimization
Nonconvex optimization
Low-rank optimization
First-order algorithms
65K05
90C30
ISSN:0022-3239, 1573-2878
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Many problems of substantial current interest in machine learning, statistics, and data science can be formulated as sparse and low-rank optimization problems. In this paper, we present the nonconvex exterior-point optimization solver (NExOS)—a first-order algorithm tailored to sparse and low-rank optimization problems. We consider the problem of minimizing a convex function over a nonconvex constraint set, where the set can be decomposed as the intersection of a compact convex set and a nonconvex set involving sparse or low-rank constraints. Unlike the convex relaxation approaches, NExOS finds a locally optimal point of the original problem by solving a sequence of penalized problems with strictly decreasing penalty parameters by exploiting the nonconvex geometry. NExOS solves each penalized problem by applying a first-order algorithm, which converges linearly to a local minimum of the corresponding penalized formulation under regularity conditions. Furthermore, the local minima of the penalized problems converge to a local minimum of the original problem as the penalty parameter goes to zero. We then implement and test NExOS on many instances from a wide variety of sparse and low-rank optimization problems, empirically demonstrating that our algorithm outperforms specialized methods.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0022-3239
1573-2878
DOI:10.1007/s10957-024-02448-9