Disentangling Representations in Restricted Boltzmann Machines without Adversaries

A goal of unsupervised machine learning is to build representations of complex high-dimensional data, with simple relations to their properties. Such disentangled representations make it easier to interpret the significant latent factors of variation in the data, as well as to generate new data with...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Physical review. X Ročník 13; číslo 2; s. 021003
Hlavní autoři: Fernandez-de-Cossio-Diaz, Jorge, Cocco, Simona, Monasson, Rémi
Médium: Journal Article
Jazyk:angličtina
Vydáno: College Park American Physical Society 01.04.2023
Témata:
Computer Science
Constraints
Cost analysis
Datasets
Discriminators
Handwriting
Image reconstruction
Ising model
Labels
Machine learning
Neural networks
Physics
Representations
Taxonomy
Training
Two dimensional models
Unsupervised learning
ISSN:2160-3308, 2160-3308
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:A goal of unsupervised machine learning is to build representations of complex high-dimensional data, with simple relations to their properties. Such disentangled representations make it easier to interpret the significant latent factors of variation in the data, as well as to generate new data with desirable features. The methods for disentangling representations often rely on an adversarial scheme, in which representations are tuned to avoid discriminators from being able to reconstruct information about the data properties (labels). Unfortunately, adversarial training is generally difficult to implement in practice. Here we propose a simple, effective way of disentangling representations without any need to train adversarial discriminators and apply our approach to Restricted Boltzmann Machines, one of the simplest representation-based generative models. Our approach relies on the introduction of adequate constraints on the weights during training, which allows us to concentrate information about labels on a small subset of latent variables. The effectiveness of the approach is illustrated with four examples: the CelebA dataset of facial images, the two-dimensional Ising model, the MNIST dataset of handwritten digits, and the taxonomy of protein families. In addition, we show how our framework allows for analytically computing the cost, in terms of the log-likelihood of the data, associated with the disentanglement of their representations.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2160-3308
2160-3308
DOI:10.1103/PhysRevX.13.021003