Dynamical System Autoencoders

Autoencoders represent a significant category of deep learning models and are widely utilized for dimensionality reduction. However, standard Autoencoders are complicated architectures that normally have several layers and many hyper-parameters that require tuning. In this paper, we introduce a new...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Proceedings (IEEE International Conference on Emerging Technologies and Factory Automation) S. 1496 - 1503
Hauptverfasser: He, Shiquan, Paffenroth, Randy, Cava, Olivia, Dunham, Cate
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 18.12.2024
Schlagworte:
autoencoder
Autoencoders
denoising
Dimensionality reduction
dynamical system autoencoder
Dynamical systems
Encoding
Iterative methods
Noise reduction
sparse autoencoder
Sparse matrices
stacked autoencoder
supervised autoencoder
Training
Tuning
ISSN:1946-0759
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Autoencoders represent a significant category of deep learning models and are widely utilized for dimensionality reduction. However, standard Autoencoders are complicated architectures that normally have several layers and many hyper-parameters that require tuning. In this paper, we introduce a new type of autoencoder that we call dynamical system autoencoder (DSAE). Similar to classic autoencoders, DSAEs can effectively handle dimensionality reduction and denoising tasks, and they demonstrate strong performance in several benchmark tasks. However, DSAEs, in some sense, have a more flexible architecture than standard AEs. In particular, in this paper we study simple DSAEs that only have a single layer. In addition, DSAEs provide several theoretical and practical advantages arising from their implementation as iterative maps, which have been well studied over several decades. Beyond the inherent simplicity of DSAEs, we also demonstrate how to use sparse matrices to reduce the number of parameters for DSAEs without sacrificing the performance of our methods. Our simulation studies indicate that DSAEs achieved better performance than the classic autoencoders when the encoding dimension or training sample size was small. Additionally, we illustrate how to use DSAEs, and denoising autoencoders in general. to nerform sunervised learning tasks.
ISSN:1946-0759
DOI:10.1109/ICMLA61862.2024.00231