Cross-covariance regularized autoencoders for nonredundant sparse feature representation

We propose a new feature representation algorithm using cross-covariance in the context of deep learning. Existing feature representation algorithms based on the sparse autoencoder and nonnegativity-constrained autoencoder tend to produce duplicative encoding and decoding receptive fields, which lea...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Neurocomputing (Amsterdam) Jg. 316; S. 49 - 58
Hauptverfasser: Chen, Jie, Wu, ZhongCheng, Zhang, Jun, Li, Fang, Li, WenJing, Wu, ZiHeng
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Elsevier B.V 17.11.2018
Schlagworte:
Autoencoder
Cross-covariance
Deep learning
Feature representation
Receptive fields
Deep learning
Receptive fields
Cross-covariance
Feature representation
Autoencoder
ISSN:0925-2312, 1872-8286
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:We propose a new feature representation algorithm using cross-covariance in the context of deep learning. Existing feature representation algorithms based on the sparse autoencoder and nonnegativity-constrained autoencoder tend to produce duplicative encoding and decoding receptive fields, which leads to feature redundancy and overfitting. We propose using the cross-covariance to regularize the feature weight vector to construct a new objective function to eliminate feature redundancy and reduce overfitting. The results from the MNIST handwritten digits dataset, the NORB normalized-uniform dataset and the Yale face dataset indicate that relative to other algorithms based on the conventional sparse autoencoder and nonnegativity-constrained autoencoder, our method can effectively eliminate feature redundancy, extract more distinctive features, and improve sparsity and reconstruction quality. Furthermore, this method improves the image classification performance and reduces the overfitting of conventional networks without adding more computational time.
ISSN:0925-2312
1872-8286
DOI:10.1016/j.neucom.2018.07.050