Efficiency of minimizing compositions of convex functions and smooth maps

We consider global efficiency of algorithms for minimizing a sum of a convex function and a composition of a Lipschitz convex function with a smooth map. The basic algorithm we rely on is the prox-linear method, which in each iteration solves a regularized subproblem formed by linearizing the smooth...

Full description

Saved in:
Bibliographic Details
Published in:Mathematical programming Vol. 178; no. 1-2; pp. 503 - 558
Main Authors: Drusvyatskiy, D., Paquette, C.
Format: Journal Article
Language:English
Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.11.2019
Springer Nature B.V
Subjects:
Algorithms
Calculus of Variations and Optimal Control; Optimization
Combinatorics
Composition
Convexity
Efficiency
Full Length Paper
Mathematical and Computational Physics
Mathematical Methods in Physics
Mathematics
Mathematics and Statistics
Mathematics of Computing
Numerical Analysis
Theoretical
Secondary 90C06
Fast gradient methods
Inexactness
Gauss–Newton
Complexity
Prox-gradient
Incremental methods
Primary 97N60
90C30
Composite minimization
Smoothing
90C25
Acceleration
ISSN:0025-5610, 1436-4646
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We consider global efficiency of algorithms for minimizing a sum of a convex function and a composition of a Lipschitz convex function with a smooth map. The basic algorithm we rely on is the prox-linear method, which in each iteration solves a regularized subproblem formed by linearizing the smooth map. When the subproblems are solved exactly, the method has efficiency O ( ε - 2 ) , akin to gradient descent for smooth minimization. We show that when the subproblems can only be solved by first-order methods, a simple combination of smoothing, the prox-linear method, and a fast-gradient scheme yields an algorithm with complexity O ~ ( ε - 3 ) . We round off the paper with an inertial prox-linear method that automatically accelerates in presence of convexity.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0025-5610
1436-4646
DOI:10.1007/s10107-018-1311-3