A convolutional neural network with sparse representation - Yang, Guoan; Yang, Junjie; Lu, Zhengzhi; Liu, Deyang (Knowledge-based systems, 2020)
This paper proposes a sparse representation layer in the feature extraction stage of a convolutional neural network (CNN). Our goal is to add sparse transforms to a target network to improve its performance without introducing an extra calculation burden. First, the proposed method was achieved by inserting the sparse representation layers into a target network’s shallow layers, and the network was trained end-to-end using a supervised learning algorithm. Second, In the forward pass the network captured the features through the convolutional layers and sparse representation layers accomplished with wavelet and shearlet transforms. Thirdly, in the backward pass the weights of the learned kernels of the network were updated through a back-propagated error, while the sparse representation layers were fixed and did not require updating. The proposed method was verified on five datasets with the task of image classification: FOOD-101, CIFAR10/100, DTD, Brodatz and ImageNet. The experimental results show that the proposed method leads to higher recognition accuracy in image classification, and the additional computational cost is relatively small compared to the baseline CNN model. •This paper proposes the hybrid of a CNN model and sparse representation.•We combine the CNN with predefined filters of wavelet and shearlet transform.•The paper theoretically explains the work procedure of the proposed network model.