Optimizing training trajectories in variational autoencoders via latent Bayesian optimization approach

Unsupervised and semi-supervised ML methods such as variational autoencoders (VAE) have become widely adopted across multiple areas of physics, chemistry, and materials sciences due to their capability in disentangling representations and ability to find latent manifolds for classification and/or re...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Machine learning: science and technology Ročník 4; číslo 1; s. 15011 - 15025
Hlavní autoři: Biswas, Arpan, Vasudevan, Rama, Ziatdinov, Maxim, Kalinin, Sergei V
Médium: Journal Article
Jazyk:angličtina
Vydáno: Bristol IOP Publishing 01.03.2023
Témata:
Bayesian analysis
Bayesian optimization
ENGINEERING
high-dimensional problem
latent space
Manifolds (mathematics)
Nanoparticles
Optimization
Representations
Topology
Training
Trajectory optimization
unsupervised learning
variation auto-encoder
variational auto-encoder
ISSN:2632-2153, 2632-2153
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Unsupervised and semi-supervised ML methods such as variational autoencoders (VAE) have become widely adopted across multiple areas of physics, chemistry, and materials sciences due to their capability in disentangling representations and ability to find latent manifolds for classification and/or regression of complex experimental data. Like other ML problems, VAEs require hyperparameter tuning, e.g. balancing the Kullback–Leibler and reconstruction terms. However, the training process and resulting manifold topology and connectivity depend not only on hyperparameters, but also their evolution during training. Because of the inefficiency of exhaustive search in a high-dimensional hyperparameter space for the expensive-to-train models, here we have explored a latent Bayesian optimization (zBO) approach for the hyperparameter trajectory optimization for the unsupervised and semi-supervised ML and demonstrated for joint-VAE with rotational invariances. We have demonstrated an application of this method for finding joint discrete and continuous rotationally invariant representations for modified national institute of standards and technology database (MNIST) and experimental data of a plasmonic nanoparticles material system. The performance of the proposed approach has been discussed extensively, where it allows for any high dimensional hyperparameter trajectory optimization of other ML models.
Bibliografie:MLST-100588.R2
ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
USDOE Office of Science (SC), Basic Energy Sciences (BES)
AC05-00OR22725; SC0019288
ISSN:2632-2153
2632-2153
DOI:10.1088/2632-2153/acb316