Submanifold-Preserving Discriminant Analysis With an Auto-Optimized Graph

Due to the multimodality of non-Gaussian data, traditional globality-preserved dimensionality reduction (DR) methods, such as linear discriminant analysis (LDA) and principal component analysis (PCA) are difficult to deal with. In this paper, we present a novel local DR framework via auto-optimized...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on cybernetics Vol. 50; no. 8; pp. 3682 - 3695
Main Authors: Nie, Feiping, Wang, Zheng, Wang, Rong, Li, Xuelong
Format: Journal Article
Language:English
Published: United States IEEE 01.08.2020
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Algorithms
Auto-optimized graph
Clustering algorithms
Constraint modelling
Data models
Data points
Direct reduction
Discriminant analysis
Embedding
Feature extraction
Germanium
Iterative methods
Kernel functions
Manifolds
Manifolds (mathematics)
Optimization
Principal component analysis
Principal components analysis
Robustness
Similarity
submanifold preserved learning
supervised and semisupervised dimensionality reduction (DR)
ISSN:2168-2267, 2168-2275, 2168-2275
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Due to the multimodality of non-Gaussian data, traditional globality-preserved dimensionality reduction (DR) methods, such as linear discriminant analysis (LDA) and principal component analysis (PCA) are difficult to deal with. In this paper, we present a novel local DR framework via auto-optimized graph embedding to extract the intrinsic submanifold structure of multimodal data. Specifically, the proposed model seeks to learn an embedding space which can preserve the local neighborhood structure by constructing a <inline-formula> <tex-math notation="LaTeX">{k} </tex-math></inline-formula>-nearest neighbors (<inline-formula> <tex-math notation="LaTeX">{k} </tex-math></inline-formula>NNs) graph on data points. Different than previous works, our model employs the <inline-formula> <tex-math notation="LaTeX">\boldsymbol {\ell }_{\boldsymbol {0}} </tex-math></inline-formula>-norm constraint and binary constraint on the similarity matrix to impose that there only be a <inline-formula> <tex-math notation="LaTeX">{k} </tex-math></inline-formula> nonzero value in each row of the similarity matrix, which can ensure the <inline-formula> <tex-math notation="LaTeX">{k} </tex-math></inline-formula>-connectivity in graph. More important, as the high-dimensional data probably contains some noises and redundant features, calculating the similarity matrix in the original space by using a kernel function is inaccurate. As a result, a mechanism of an auto-optimized graph is derived in the proposed model. Concretely, we learn the embedding space and similarity matrix simultaneously. In other words, the selection of neighbors is automatically executed in the optimal subspace rather than in the original space when the algorithm reaches convergence, which can alleviate the affect of noises and improve the robustness of the proposed model. In addition, four supervised and semisupervised local DR methods are derived by the proposed framework which can extract the discriminative features while preserving the submanifold structure of data. Last but not least, since two variables need to be optimized simultaneously in the proposed methods, and the constraints on the similarity matrix are difficult to satisfy, which is an NP-hard problem. Consequently, an efficient iterative optimization algorithm is introduced to solve the proposed problems. Extensive experiments conducted on synthetic data and several real-world datasets have demonstrated the advantages of the proposed methods in robustness and recognition accuracy.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:2168-2267
2168-2275
2168-2275
DOI:10.1109/TCYB.2019.2910751