Dynamical System Autoencoders

Autoencoders represent a significant category of deep learning models and are widely utilized for dimensionality reduction. However, standard Autoencoders are complicated architectures that normally have several layers and many hyper-parameters that require tuning. In this paper, we introduce a new...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings (IEEE International Conference on Emerging Technologies and Factory Automation) pp. 1496 - 1503
Main Authors: He, Shiquan, Paffenroth, Randy, Cava, Olivia, Dunham, Cate
Format: Conference Proceeding
Language:English
Published: IEEE 18.12.2024
Subjects:
autoencoder
Autoencoders
denoising
Dimensionality reduction
dynamical system autoencoder
Dynamical systems
Encoding
Iterative methods
Noise reduction
sparse autoencoder
Sparse matrices
stacked autoencoder
supervised autoencoder
Training
Tuning
ISSN:1946-0759
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Autoencoders represent a significant category of deep learning models and are widely utilized for dimensionality reduction. However, standard Autoencoders are complicated architectures that normally have several layers and many hyper-parameters that require tuning. In this paper, we introduce a new type of autoencoder that we call dynamical system autoencoder (DSAE). Similar to classic autoencoders, DSAEs can effectively handle dimensionality reduction and denoising tasks, and they demonstrate strong performance in several benchmark tasks. However, DSAEs, in some sense, have a more flexible architecture than standard AEs. In particular, in this paper we study simple DSAEs that only have a single layer. In addition, DSAEs provide several theoretical and practical advantages arising from their implementation as iterative maps, which have been well studied over several decades. Beyond the inherent simplicity of DSAEs, we also demonstrate how to use sparse matrices to reduce the number of parameters for DSAEs without sacrificing the performance of our methods. Our simulation studies indicate that DSAEs achieved better performance than the classic autoencoders when the encoding dimension or training sample size was small. Additionally, we illustrate how to use DSAEs, and denoising autoencoders in general. to nerform sunervised learning tasks.
ISSN:1946-0759
DOI:10.1109/ICMLA61862.2024.00231