A Regularized Semi-Smooth Newton Method with Projection Steps for Composite Convex Programs

The goal of this paper is to study approaches to bridge the gap between first-order and second-order type methods for composite convex programs. Our key observations are: (1) Many well-known operator splitting methods, such as forward–backward splitting and Douglas–Rachford splitting, actually defin...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Journal of scientific computing Jg. 76; H. 1; S. 364 - 389
Hauptverfasser: Xiao, Xiantao, Li, Yongfeng, Wen, Zaiwen, Zhang, Liwei
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York Springer US 01.07.2018
Springer Nature B.V
Schlagworte:
Algorithms
Computational Mathematics and Numerical Analysis
Convergence
Convex analysis
Hyperplanes
Jacobi matrix method
Jacobian matrix
Machine learning
Mapping
Mathematical and Computational Engineering
Mathematical and Computational Physics
Mathematics
Mathematics and Statistics
Methods
Newton methods
Nonlinear equations
Optimization
Partial differential equations
Regularization
Splitting
Theoretical
Semi-smoothness
Operator splitting methods
90C30
65K05
Proximal mapping
Newton method
Composite convex programs
ISSN:0885-7474, 1573-7691
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The goal of this paper is to study approaches to bridge the gap between first-order and second-order type methods for composite convex programs. Our key observations are: (1) Many well-known operator splitting methods, such as forward–backward splitting and Douglas–Rachford splitting, actually define a fixed-point mapping; (2) The optimal solutions of the composite convex program and the solutions of a system of nonlinear equations derived from the fixed-point mapping are equivalent. Solving this kind of system of nonlinear equations enables us to develop second-order type methods. These nonlinear equations may be non-differentiable, but they are often semi-smooth and their generalized Jacobian matrix is positive semidefinite due to monotonicity. By combining with a regularization approach and a known hyperplane projection technique, we propose an adaptive semi-smooth Newton method and establish its convergence to global optimality. Preliminary numerical results on ℓ 1 -minimization problems demonstrate that our second-order type algorithms are able to achieve superlinear or quadratic convergence.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0885-7474
1573-7691
DOI:10.1007/s10915-017-0624-3