A Regularized Semi-Smooth Newton Method with Projection Steps for Composite Convex Programs

The goal of this paper is to study approaches to bridge the gap between first-order and second-order type methods for composite convex programs. Our key observations are: (1) Many well-known operator splitting methods, such as forward–backward splitting and Douglas–Rachford splitting, actually defin...

Full description

Saved in:
Bibliographic Details
Published in:Journal of scientific computing Vol. 76; no. 1; pp. 364 - 389
Main Authors: Xiao, Xiantao, Li, Yongfeng, Wen, Zaiwen, Zhang, Liwei
Format: Journal Article
Language:English
Published: New York Springer US 01.07.2018
Springer Nature B.V
Subjects:
Algorithms
Computational Mathematics and Numerical Analysis
Convergence
Convex analysis
Hyperplanes
Jacobi matrix method
Jacobian matrix
Machine learning
Mapping
Mathematical and Computational Engineering
Mathematical and Computational Physics
Mathematics
Mathematics and Statistics
Methods
Newton methods
Nonlinear equations
Optimization
Partial differential equations
Regularization
Splitting
Theoretical
Semi-smoothness
Operator splitting methods
90C30
65K05
Proximal mapping
Newton method
Composite convex programs
ISSN:0885-7474, 1573-7691
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The goal of this paper is to study approaches to bridge the gap between first-order and second-order type methods for composite convex programs. Our key observations are: (1) Many well-known operator splitting methods, such as forward–backward splitting and Douglas–Rachford splitting, actually define a fixed-point mapping; (2) The optimal solutions of the composite convex program and the solutions of a system of nonlinear equations derived from the fixed-point mapping are equivalent. Solving this kind of system of nonlinear equations enables us to develop second-order type methods. These nonlinear equations may be non-differentiable, but they are often semi-smooth and their generalized Jacobian matrix is positive semidefinite due to monotonicity. By combining with a regularization approach and a known hyperplane projection technique, we propose an adaptive semi-smooth Newton method and establish its convergence to global optimality. Preliminary numerical results on ℓ 1 -minimization problems demonstrate that our second-order type algorithms are able to achieve superlinear or quadratic convergence.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0885-7474
1573-7691
DOI:10.1007/s10915-017-0624-3