Theoretical insights on the pre-image resolution in machine learning

While many nonlinear pattern recognition and data mining tasks rely on embedding the data into a latent space, one often needs to extract the patterns in the input space. Estimating the inverse of the nonlinear embedding is the so-called pre-image problem. Several strategies have been proposed to ad...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Pattern recognition Ročník 156; s. 110800
Hlavný autor: Honeine, Paul
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Elsevier Ltd 01.12.2024
Elsevier
Predmet:
Artificial Intelligence
Computer Science
Fixed-point iteration
Machine Learning
Majorize-minimization algorithm
Newton’s method
Pattern recognition
Pre-image problem
Majorize-minimization algorithm
Newton’s method
Fixed-point iteration
Pattern recognition
Machine learning
Pre-image problem
Pattern Recognition
Newton's Method
Fixed-point Iteration
Majorize-Minimization Algorithm
Machine Learning
Pre-image Problem
ISSN:0031-3203
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:While many nonlinear pattern recognition and data mining tasks rely on embedding the data into a latent space, one often needs to extract the patterns in the input space. Estimating the inverse of the nonlinear embedding is the so-called pre-image problem. Several strategies have been proposed to address the estimation of the pre-image; However, there are no theoretical results so far to understand the pre-image problem and its resolution. In this paper, we provide theoretical underpinnings of the resolution of the pre-image problem in Machine Learning. These theoretical results are on the gradient descent optimization, the fixed-point iteration algorithm and Newton’s method. We provide sufficient conditions on the convexity/nonconvexity of the pre-image problem. Moreover, we show that the fixed-point iteration is a Newton update and prove that it is a Majorize-Minimization (MM) algorithm where the surrogate function is a quadratic function. These theoretical results are derived for the wide classes of radial kernels and projective kernels. We also provide other insights by connecting the resolution of this problem to the gradient density estimation problem with the so-called mean shift algorithm. •Solid foundations on the resolution of the pre-image problem in Machine Learning•Relationship between the fixed-point iteration technique and Newton’s method•Fixed-point iteration is a Majorize-Minimization algorithm•General theoretical results for the wide classes of radial and projective kernels
ISSN:0031-3203
DOI:10.1016/j.patcog.2024.110800