Optimizing training trajectories in variational autoencoders via latent Bayesian optimization approach

Unsupervised and semi-supervised ML methods such as variational autoencoders (VAE) have become widely adopted across multiple areas of physics, chemistry, and materials sciences due to their capability in disentangling representations and ability to find latent manifolds for classification and/or re...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Machine learning: science and technology Jg. 4; H. 1; S. 15011 - 15025
Hauptverfasser: Biswas, Arpan, Vasudevan, Rama, Ziatdinov, Maxim, Kalinin, Sergei V
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Bristol IOP Publishing 01.03.2023
Schlagworte:
Bayesian analysis
Bayesian optimization
ENGINEERING
high-dimensional problem
latent space
Manifolds (mathematics)
Nanoparticles
Optimization
Representations
Topology
Training
Trajectory optimization
unsupervised learning
variation auto-encoder
variational auto-encoder
ISSN:2632-2153, 2632-2153
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Unsupervised and semi-supervised ML methods such as variational autoencoders (VAE) have become widely adopted across multiple areas of physics, chemistry, and materials sciences due to their capability in disentangling representations and ability to find latent manifolds for classification and/or regression of complex experimental data. Like other ML problems, VAEs require hyperparameter tuning, e.g. balancing the Kullback–Leibler and reconstruction terms. However, the training process and resulting manifold topology and connectivity depend not only on hyperparameters, but also their evolution during training. Because of the inefficiency of exhaustive search in a high-dimensional hyperparameter space for the expensive-to-train models, here we have explored a latent Bayesian optimization (zBO) approach for the hyperparameter trajectory optimization for the unsupervised and semi-supervised ML and demonstrated for joint-VAE with rotational invariances. We have demonstrated an application of this method for finding joint discrete and continuous rotationally invariant representations for modified national institute of standards and technology database (MNIST) and experimental data of a plasmonic nanoparticles material system. The performance of the proposed approach has been discussed extensively, where it allows for any high dimensional hyperparameter trajectory optimization of other ML models.
Bibliographie:MLST-100588.R2
ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
USDOE Office of Science (SC), Basic Energy Sciences (BES)
AC05-00OR22725; SC0019288
ISSN:2632-2153
2632-2153
DOI:10.1088/2632-2153/acb316