Cross-covariance regularized autoencoders for nonredundant sparse feature representation

We propose a new feature representation algorithm using cross-covariance in the context of deep learning. Existing feature representation algorithms based on the sparse autoencoder and nonnegativity-constrained autoencoder tend to produce duplicative encoding and decoding receptive fields, which lea...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Neurocomputing (Amsterdam) Ročník 316; s. 49 - 58
Hlavní autori: Chen, Jie, Wu, ZhongCheng, Zhang, Jun, Li, Fang, Li, WenJing, Wu, ZiHeng
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Elsevier B.V 17.11.2018
Predmet:
Autoencoder
Cross-covariance
Deep learning
Feature representation
Receptive fields
Deep learning
Receptive fields
Cross-covariance
Feature representation
Autoencoder
ISSN:0925-2312, 1872-8286
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:We propose a new feature representation algorithm using cross-covariance in the context of deep learning. Existing feature representation algorithms based on the sparse autoencoder and nonnegativity-constrained autoencoder tend to produce duplicative encoding and decoding receptive fields, which leads to feature redundancy and overfitting. We propose using the cross-covariance to regularize the feature weight vector to construct a new objective function to eliminate feature redundancy and reduce overfitting. The results from the MNIST handwritten digits dataset, the NORB normalized-uniform dataset and the Yale face dataset indicate that relative to other algorithms based on the conventional sparse autoencoder and nonnegativity-constrained autoencoder, our method can effectively eliminate feature redundancy, extract more distinctive features, and improve sparsity and reconstruction quality. Furthermore, this method improves the image classification performance and reduces the overfitting of conventional networks without adding more computational time.
ISSN:0925-2312
1872-8286
DOI:10.1016/j.neucom.2018.07.050