Novel DCA based algorithms for a special class of nonconvex problems with application in machine learning

•We address the problem of minimizing the sum of a nonconvex, differentiable function and composite functions by DC (Difference of Convex functions) programming and DCA (DC Algorithm).•We first develop a standard DCA scheme especially dealing with the very specific structure of this problem.•Further...

Full description

Saved in:
Bibliographic Details
Published in:Applied mathematics and computation Vol. 409; p. 125904
Main Authors: Le Thi, Hoai An, Le, Hoai Minh, Phan, Duy Nhat, Tran, Bach
Format: Journal Article
Language:English
Published: Elsevier Inc 15.11.2021
Elsevier
Subjects:
Accelerated DCA
Accelerated DCA-Like
Computer Science
DC programming
DCA
DCA-Like
Dimensionality reduction
Sum of nonconvex function and composite functions
t-distributed Stochastic Neighbor Embedding (t-SNE)
Dimensionality reduction
DCA
Accelerated DCA
Sum of nonconvex function and composite functions
DC programming
DCA-Like
Accelerated DCA-Like
t-distributed Stochastic Neighbor Embedding (t-SNE)
T-SNE
Accelerated DCA-like
DCA-like
ISSN:0096-3003, 1873-5649
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:•We address the problem of minimizing the sum of a nonconvex, differentiable function and composite functions by DC (Difference of Convex functions) programming and DCA (DC Algorithm).•We first develop a standard DCA scheme especially dealing with the very specific structure of this problem.•Furthermore, we extend DCA to give rise to the so-named DCA-Like, which is based on a new and efficient way to approximate the DC objective function without knowing a DC decomposition.•We further improve DCA based algorithms by incorporating the Nesterov’s acceleration technique into them.•Finally, we investigate the proposed algorithms for an important problem in machine learning: the t-distributed stochastic neighbor embedding. We address the problem of minimizing the sum of a nonconvex, differentiable function and composite functions by DC (Difference of Convex functions) programming and DCA (DC Algorithm), powerful tools of nonconvex optimization. The main idea of DCA relies on DC decompositions of the objective function, it consists in approximating a DC (nonconvex) program by a sequence of convex ones. We first develop a standard DCA scheme especially dealing with the very specific structure of this problem. Furthermore, we extend DCA to give rise to the so-named DCA-Like, which is based on a new and efficient way to approximate the DC objective function without knowing a DC decomposition. We further improve DCA based algorithms by incorporating the Nesterov’s acceleration technique into them. The convergence properties and the convergence rate under Kurdyka-Łojasiewicz assumption of extended DCAs are rigorously studied. We prove that DCA-Like and the accelerated versions subsequently converge from every initial point to a critical point of the considered problem. Finally, we investigate the proposed algorithms for an important problem in machine learning: the t-distributed stochastic neighbor embedding. Numerical experiments on several benchmark datasets illustrate the efficiency of our algorithms.
ISSN:0096-3003
1873-5649
DOI:10.1016/j.amc.2020.125904