Accelerated sparse nonnegative matrix factorization for unsupervised feature learning

•We improved SNMF with implicit sparse constraints which are the L1-norm of coefficient matrix and L2-norm of basis matrix.•A subproblem is transformed into a convex optimization model solved by DG, another one is equivalent to FISTA by Lip. Cond.•We obtain the closed-form iteration form of each sub...

Full description

Saved in:
Bibliographic Details
Published in:Pattern recognition letters Vol. 156; pp. 46 - 52
Main Authors: Xie, Ting, Zhang, Hua, Liu, Ruihua, Xiao, Hanguang
Format: Journal Article
Language:English
Published: Amsterdam Elsevier B.V 01.04.2022
Elsevier Science Ltd
Subjects:
Algorithms
Closed form solutions
Clustering
Exact solutions
Factorization
Iterative algorithms
Iterative methods
Iterative solution
Machine learning
Mathematical analysis
Nonnegative matrix factorization
Norms
Sparse
Nonnegative matrix factorization
41A10
65D05
65D17
Sparse
Clustering
41A05
ISSN:0167-8655, 1872-7344
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:•We improved SNMF with implicit sparse constraints which are the L1-norm of coefficient matrix and L2-norm of basis matrix.•A subproblem is transformed into a convex optimization model solved by DG, another one is equivalent to FISTA by Lip. Cond.•We obtain the closed-form iteration form of each sub-problem by ADMM and learn the cluster assignments by resulting features.•We analyzed the convergence of algorithm in theory and proved that the convergence point is KKT point of equivalent model.•We analyzed the algorithm complexity and presented the performance of running time, CA and MI under the SVD initialization. Sparse Nonnegative Matrix Factorization (SNMF) is a fundamental unsupervised representation learning technique, and it represents low-dimensional features of a data set and lends itself to a clustering interpretation. However, the model and algorithm of SNMF have some shortcomings. In this work, we created a clustering method by improving the SNMF model and its Alternating Direction Multiplier Method acceleration algorithm. A novel, fast and closed-form iterative solution is proposed for SNMF with implicit sparse constraints which are L1 and L2 norms of the coefficient and basis matrixes, respectively. A low-dimensional feature space is also proposed as result of the closed-form iteration formats of each sub-problem obtained by variable splitting. In addition, the convergence points of the presented iterative algorithms are stationary points of the model. Finally, numerical experiments show that the improved algorithm is comparable to the sate-of-the-art methods in data clustering.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0167-8655
1872-7344
DOI:10.1016/j.patrec.2022.01.020