Online Kernel Principal Component Analysis: A Reduced-Order Model

Kernel principal component analysis (kernel-PCA) is an elegant nonlinear extension of one of the most used data analysis and dimensionality reduction techniques, the principal component analysis. In this paper, we propose an online algorithm for kernel-PCA. To this end, we examine a kernel-based ver...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on pattern analysis and machine intelligence Vol. 34; no. 9; pp. 1814 - 1826
Main Author: Honeine, P.
Format: Journal Article
Language:English
Published: Los Alamitos, CA IEEE 01.09.2012
IEEE Computer Society
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Institute of Electrical and Electronics Engineers
Subjects:
Algorithm design and analysis
Algorithmics. Computability. Computer arithmetics
Algorithms
Applied sciences
Artificial intelligence
Computer Science
Computer science; control theory; systems
Data models
Data processing. List processing. Character string processing
Dictionaries
Eigenvalues and eigenfunctions
Engineering Sciences
Exact sciences and technology
Intelligence
Kernel
Kernels
Machine Learning
Memory organisation. Data processing
Oja's rule
On-line systems
Online
online algorithm
Pattern analysis
Pattern recognition. Digital image processing. Computational geometry
Principal component analysis
recursive algorithm
reproducing kernel
Signal and Image processing
Software
Studies
Theoretical computing
Training data
Data analysis
On line
online algorithm
Error bound
Iterative method
machine learning
reproducing kernel
Recursive algorithm
Modeling
Data reduction
Kernel method
Oja's rule
Dimension reduction
Upper bound
Experimental result
Efficiency
Optical character recognition
Reduced order systems
Manuscript character
Artificial intelligence
Principal component analysis
Algorithm design and analysis
Dictionaries
sparsity
function approximation
Training data
recursive algorithm
principal function approximation
reduced order systems
Oja rule
Eigenvalues and eigenfunctions
Kernel
data analysis
adaptive filtering
synthetic data set
online kernel principal component analysis
kernel-based machines
pre-image problem
dimensionality reduction techniques
classical kernel-PCA
handwritten digit image
non-stationarity
reduced-order model
iterative kernel-PCA
linear principal axe extraction
Data models
Index Terms-Principal component analysis
ISSN:0162-8828, 1939-3539, 2160-9292, 1939-3539
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Kernel principal component analysis (kernel-PCA) is an elegant nonlinear extension of one of the most used data analysis and dimensionality reduction techniques, the principal component analysis. In this paper, we propose an online algorithm for kernel-PCA. To this end, we examine a kernel-based version of Oja's rule, initially put forward to extract a linear principal axe. As with most kernel-based machines, the model order equals the number of available observations. To provide an online scheme, we propose to control the model order. We discuss theoretical results, such as an upper bound on the error of approximating the principal functions with the reduced-order model. We derive a recursive algorithm to discover the first principal axis, and extend it to multiple axes. Experimental results demonstrate the effectiveness of the proposed approach, both on synthetic data set and on images of handwritten digits, with comparison to classical kernel-PCA and iterative kernel-PCA.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ObjectType-Article-2
ObjectType-Feature-1
ISSN:0162-8828
1939-3539
2160-9292
1939-3539
DOI:10.1109/TPAMI.2011.270