Physics and chemistry from parsimonious representations: image analysis via invariant variational autoencoders

Electron, optical, and scanning probe microscopy methods are generating ever increasing volume of image data containing information on atomic and mesoscale structures and functionalities. This necessitates the development of the machine learning methods for discovery of physical and chemical phenome...

Full description

Saved in:
Bibliographic Details
Published in:npj computational materials Vol. 10; no. 1; pp. 183 - 19
Main Authors: Valleti, Mani, Ziatdinov, Maxim, Liu, Yongtao, Kalinin, Sergei V.
Format: Journal Article
Language:English
Published: London Nature Publishing Group UK 14.08.2024
Nature Publishing Group
Nature Portfolio
Subjects:
639/301
639/925/918
Broken symmetry
Characterization and Evaluation of Materials
Chemistry and Materials Science
Computational Intelligence
Data analysis
Datasets
graphene
Image analysis
Image processing
Invariants
Machine learning
MATERIALS SCIENCE
Mathematical and Computational Engineering
Mathematical and Computational Physics
Mathematical Modeling and Industrial Mathematics
Microscopy
Nanoparticles
Representations
Scanning probe microscopy
Scanning transmission electron microscopy
Scanning tunneling microscopy
Theoretical
Transmission electron microscopy
ISSN:2057-3960, 2057-3960
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Electron, optical, and scanning probe microscopy methods are generating ever increasing volume of image data containing information on atomic and mesoscale structures and functionalities. This necessitates the development of the machine learning methods for discovery of physical and chemical phenomena from the data, such as manifestations of symmetry breaking phenomena in electron and scanning tunneling microscopy images, or variability of the nanoparticles. Variational autoencoders (VAEs) are emerging as a powerful paradigm for the unsupervised data analysis, allowing to disentangle the factors of variability and discover optimal parsimonious representation. Here, we summarize recent developments in VAEs, covering the basic principles and intuition behind the VAEs. The invariant VAEs are introduced as an approach to accommodate scale and translation invariances present in imaging data and separate known factors of variations from the ones to be discovered. We further describe the opportunities enabled by the control over VAE architecture, including conditional, semi-supervised, and joint VAEs. Several case studies of VAE applications for toy models and experimental datasets in Scanning Transmission Electron Microscopy are discussed, emphasizing the deep connection between VAE and basic physical principles. Python codes and datasets discussed in this article are available at https://github.com/saimani5/VAE-tutorials and can be used by researchers as an application guide when applying these to their own datasets.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
USDOE Office of Science (SC), Basic Energy Sciences (BES). Scientific User Facilities (SUF)
USDOE Laboratory Directed Research and Development (LDRD) Program
None
AC05-00OR22725; SC0019288
ISSN:2057-3960
2057-3960
DOI:10.1038/s41524-024-01250-5