Theoretical insights on the pre-image resolution in machine learning

While many nonlinear pattern recognition and data mining tasks rely on embedding the data into a latent space, one often needs to extract the patterns in the input space. Estimating the inverse of the nonlinear embedding is the so-called pre-image problem. Several strategies have been proposed to ad...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Pattern recognition Jg. 156; S. 110800
1. Verfasser: Honeine, Paul
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Elsevier Ltd 01.12.2024
Elsevier
Schlagworte:
Artificial Intelligence
Computer Science
Fixed-point iteration
Machine Learning
Majorize-minimization algorithm
Newton’s method
Pattern recognition
Pre-image problem
Majorize-minimization algorithm
Newton’s method
Fixed-point iteration
Pattern recognition
Machine learning
Pre-image problem
Pattern Recognition
Newton's Method
Fixed-point Iteration
Majorize-Minimization Algorithm
Machine Learning
Pre-image Problem
ISSN:0031-3203
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:While many nonlinear pattern recognition and data mining tasks rely on embedding the data into a latent space, one often needs to extract the patterns in the input space. Estimating the inverse of the nonlinear embedding is the so-called pre-image problem. Several strategies have been proposed to address the estimation of the pre-image; However, there are no theoretical results so far to understand the pre-image problem and its resolution. In this paper, we provide theoretical underpinnings of the resolution of the pre-image problem in Machine Learning. These theoretical results are on the gradient descent optimization, the fixed-point iteration algorithm and Newton’s method. We provide sufficient conditions on the convexity/nonconvexity of the pre-image problem. Moreover, we show that the fixed-point iteration is a Newton update and prove that it is a Majorize-Minimization (MM) algorithm where the surrogate function is a quadratic function. These theoretical results are derived for the wide classes of radial kernels and projective kernels. We also provide other insights by connecting the resolution of this problem to the gradient density estimation problem with the so-called mean shift algorithm. •Solid foundations on the resolution of the pre-image problem in Machine Learning•Relationship between the fixed-point iteration technique and Newton’s method•Fixed-point iteration is a Majorize-Minimization algorithm•General theoretical results for the wide classes of radial and projective kernels
ISSN:0031-3203
DOI:10.1016/j.patcog.2024.110800