Local and global convergence of a general inertial proximal splitting scheme for minimizing composite functions

This paper is concerned with convex composite minimization problems in a Hilbert space. In these problems, the objective is the sum of two closed, proper, and convex functions where one is smooth and the other admits a computationally inexpensive proximal operator. We analyze a family of generalized...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Computational optimization and applications Jg. 67; H. 2; S. 259 - 292
Hauptverfasser: Johnstone, Patrick R., Moulin, Pierre
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York Springer US 01.06.2017
Springer Nature B.V
Schlagworte:
Algorithms
Convergence
Convex analysis
Convex and Discrete Geometry
Hilbert space
Iterative methods
Management Science
Manifolds
Mathematical analysis
Mathematical models
Mathematics
Mathematics and Statistics
Methods
Operations Research
Operations Research/Decision Theory
Optimization
Problem solving
Splitting
Statistics
Studies
Theorems
FISTA
Inertial forward-backward splitting
Inertial proximal gradient
Local linear convergence
Momentum methods
Lasso
ISSN:0926-6003, 1573-2894
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:This paper is concerned with convex composite minimization problems in a Hilbert space. In these problems, the objective is the sum of two closed, proper, and convex functions where one is smooth and the other admits a computationally inexpensive proximal operator. We analyze a family of generalized inertial proximal splitting algorithms (GIPSA) for solving such problems. We establish weak convergence of the generated sequence when the minimum is attained. Our analysis unifies and extends several previous results. We then focus on ℓ 1 -regularized optimization, which is the ubiquitous special case where the nonsmooth term is the ℓ 1 -norm. For certain parameter choices, GIPSA is amenable to a local analysis for this problem. For these choices we show that GIPSA achieves finite “active manifold identification”, i.e. convergence in a finite number of iterations to the optimal support and sign, after which GIPSA reduces to minimizing a local smooth function. We prove local linear convergence under either restricted strong convexity or a strict complementarity condition. We determine the rate in terms of the inertia, stepsize, and local curvature. Our local analysis is applicable to certain recent variants of the Fast Iterative Shrinkage–Thresholding Algorithm (FISTA), for which we establish active manifold identification and local linear convergence. Based on our analysis we propose a momentum restart scheme in these FISTA variants to obtain the optimal local linear convergence rate while maintaining desirable global properties.
Bibliographie:SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
ObjectType-Article-1
ObjectType-Feature-2
content type line 23
ISSN:0926-6003
1573-2894
DOI:10.1007/s10589-017-9896-7