Conditional gradient algorithms for norm-regularized smooth convex optimization

Motivated by some applications in signal processing and machine learning, we consider two convex optimization problems where, given a cone K , a norm ‖ · ‖ and a smooth convex function f , we want either (1) to minimize the norm over the intersection of the cone and a level set of f , or (2) to mini...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	Mathematical programming Jg. 152; H. 1-2; S. 75 - 112
Hauptverfasser:	Harchaoui, Zaid, Juditsky, Anatoli, Nemirovski, Arkadi
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	Berlin/Heidelberg Springer Berlin Heidelberg 01.08.2015 Springer Nature B.V
Schlagworte:	Algorithms Artificial intelligence Calculus of Variations and Optimal Control; Optimization Combinatorics Computational geometry Convex analysis Convexity Estimates Full Length Paper Geometry Intersections Machine learning Mathematical and Computational Physics Mathematical Methods in Physics Mathematical models Mathematical programming Mathematics Mathematics and Statistics Mathematics of Computing Norms Numerical Analysis Optimization Signal processing Studies Theoretical Nuclear Norm Penalized Minimization Problem Lipschitz Continuous Gradient Discrete Gradient Field Conditional Gradient Algorithm
ISSN:	0025-5610, 1436-4646
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Beschreibung
Zusammenfassung:	Motivated by some applications in signal processing and machine learning, we consider two convex optimization problems where, given a cone K , a norm ‖ · ‖ and a smooth convex function f , we want either (1) to minimize the norm over the intersection of the cone and a level set of f , or (2) to minimize over the cone the sum of f and a multiple of the norm. We focus on the case where (a) the dimension of the problem is too large to allow for interior point algorithms, (b) ‖ · ‖ is “too complicated” to allow for computationally cheap Bregman projections required in the first-order proximal gradient algorithms. On the other hand, we assume that it is relatively easy to minimize linear forms over the intersection of K and the unit ‖ · ‖ -ball. Motivating examples are given by the nuclear norm with K being the entire space of matrices, or the positive semidefinite cone in the space of symmetric matrices, and the Total Variation norm on the space of 2D images. We discuss versions of the Conditional Gradient algorithm capable to handle our problems of interest, provide the related theoretical efficiency estimates and outline some applications.
Bibliographie:	SourceType-Scholarly Journals-1 ObjectType-Feature-1 content type line 14 ObjectType-Article-1 ObjectType-Feature-2 content type line 23
ISSN:	0025-5610 1436-4646
DOI:	10.1007/s10107-014-0778-9