Learning Gaussian-Bernoulli RBMs Using Difference of Convex Functions Optimization

The Gaussian-Bernoulli restricted Boltzmann machine (GB-RBM) is a useful generative model that captures meaningful features from the given <inline-formula> <tex-math notation="LaTeX">n </tex-math></inline-formula>-dimensional continuous data. The difficulties associ...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:IEEE transaction on neural networks and learning systems Ročník 33; číslo 10; s. 5728 - 5738
Hlavní autori: Upadhya, Vidyadhar, Sastry, P. S.
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: United States IEEE 01.10.2022
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:
Algorithms
Approximation algorithms
Computational modeling
Contrastive divergence (CD)
Convex analysis
Convex functions
Data models
difference of convex programming (DCP)
Gaussian > Bernoulli RBM (GB-RBM)
Learning
Machine learning
Markov processes
Optimization
Programming
restricted Boltzmann machine (RBM)
Stochasticity
Training
ISSN:2162-237X, 2162-2388, 2162-2388
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:The Gaussian-Bernoulli restricted Boltzmann machine (GB-RBM) is a useful generative model that captures meaningful features from the given <inline-formula> <tex-math notation="LaTeX">n </tex-math></inline-formula>-dimensional continuous data. The difficulties associated with learning GB-RBM are reported extensively in earlier studies. They indicate that the training of the GB-RBM using the current standard algorithms, namely contrastive divergence (CD) and persistent contrastive divergence (PCD), needs a carefully chosen small learning rate to avoid divergence which, in turn, results in slow learning. In this work, we alleviate such difficulties by showing that the negative log-likelihood for a GB-RBM can be expressed as a difference of convex functions if we keep the variance of the conditional distribution of visible units (given hidden unit states) and the biases of the visible units, constant. Using this, we propose a stochastic difference of convex (DC) functions programming (S-DCP) algorithm for learning the GB-RBM. We present extensive empirical studies on several benchmark data sets to validate the performance of this S-DCP algorithm. It is seen that S-DCP is better than the CD and PCD algorithms in terms of speed of learning and the quality of the generative model learned.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:2162-237X
2162-2388
2162-2388
DOI:10.1109/TNNLS.2021.3071358