A Unifying Representer Theorem for Inverse Problems and Machine Learning

Regularization addresses the ill-posedness of the training problem in machine learning or the reconstruction of a signal from a limited number of measurements. The method is applicable whenever the problem is formulated as an optimization task. The standard strategy consists in augmenting the origin...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Foundations of computational mathematics Jg. 21; H. 4; S. 941 - 960
1. Verfasser: Unser, Michael
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York Springer US 01.08.2021
Springer Nature B.V
Schlagworte:
Applications of Mathematics
Banach spaces
Computer Science
Economics
Inverse problems
Linear and Multilinear Algebras
Machine learning
Math Applications in Computer Science
Mathematics
Mathematics and Statistics
Matrix Theory
Numerical Analysis
Optimization
Regularization
Theorems
68T05
Convex optimization
Machine learning
Representer theorem
65J20
Banach space
Regularization
46N10
47A52
Inverse problem
ISSN:1615-3375, 1615-3383
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Regularization addresses the ill-posedness of the training problem in machine learning or the reconstruction of a signal from a limited number of measurements. The method is applicable whenever the problem is formulated as an optimization task. The standard strategy consists in augmenting the original cost functional by an energy that penalizes solutions with undesirable behavior. The effect of regularization is very well understood when the penalty involves a Hilbertian norm. Another popular configuration is the use of an ℓ 1 -norm (or some variant thereof) that favors sparse solutions. In this paper, we propose a higher-level formulation of regularization within the context of Banach spaces. We present a general representer theorem that characterizes the solutions of a remarkably broad class of optimization problems. We then use our theorem to retrieve a number of known results in the literature such as the celebrated representer theorem of machine leaning for RKHS, Tikhonov regularization, representer theorems for sparsity promoting functionals, the recovery of spikes, as well as a few new ones.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1615-3375
1615-3383
DOI:10.1007/s10208-020-09472-x