An Alternating Augmented Lagrangian method for constrained nonconvex optimization

We consider the problem of minimizing a smooth nonconvex function over a structured convex feasible set, that is, defined by two sets of constraints that are easy to treat when considered separately. In order to exploit the structure of the problem, we define an equivalent formulation by duplicating...

Full description

Saved in:
Bibliographic Details
Published in:Optimization methods & software Vol. 35; no. 3; pp. 502 - 520
Main Authors: Galvan, G., Lapucci, M., Levato, T., Sciandrone, M.
Format: Journal Article
Language:English
Published: Abingdon Taylor & Francis 03.05.2020
Taylor & Francis Ltd
Subjects:
Algorithms
Alternating direction method of multipliers
augmented Lagrangian
Constraints
Convexity
decomposition
Exact solutions
global convergence
Machine learning
nonconvex objective function
Nonlinear programming
Optimization
penalty method
Queuing theory
ISSN:1055-6788, 1029-4937
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We consider the problem of minimizing a smooth nonconvex function over a structured convex feasible set, that is, defined by two sets of constraints that are easy to treat when considered separately. In order to exploit the structure of the problem, we define an equivalent formulation by duplicating the variables and we consider the augmented Lagrangian of this latter formulation. Following the idea of the Alternating Direction Method of Multipliers (ADMM), we propose an algorithm where a two-blocks decomposition method is embedded within an augmented Lagrangian framework. The peculiarities of the proposed algorithm are the following: (1) the computation of the exact solution of a possibly nonconvex subproblem is not required; (2) the penalty parameter is iteratively updated once an approximated stationary point of the augmented Lagrangian is determined. Global convergence results are stated under mild assumptions and without requiring convexity of the objective function. Although the primary aim of the paper is theoretical, we perform numerical experiments on a nonconvex problem arising in machine learning, and the obtained results show the practical advantages of the proposed approach with respect to classical ADMM.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1055-6788
1029-4937
DOI:10.1080/10556788.2019.1576177