Linear Approximation of Deep Neural Networks for Efficient Inference on Video Data

Sequential data such as video are characterized by spatio-temporal correlations. As of yet, few deep learning algorithms exploit them to decrease the often massive cost during inference. This work leverages correlations in video data to linearize part of a deep neural network and thus reduce its siz...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:2019 27th European Signal Processing Conference (EUSIPCO) s. 1 - 5
Hlavní autori: Rueckauer, Bodo, Liu, Shih-Chii
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: EURASIP 01.09.2019
Predmet:
Biological neural networks
compression
Convolution
Correlation
Deep neural networks
linearization
Neurons
Noise reduction
sequential data
Task analysis
Taylor series
video
ISSN:2076-1465
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Sequential data such as video are characterized by spatio-temporal correlations. As of yet, few deep learning algorithms exploit them to decrease the often massive cost during inference. This work leverages correlations in video data to linearize part of a deep neural network and thus reduce its size and computational cost. Drawing upon the simplicity of the typically used rectifier activation function, we replace the ReLU function by dynamically updating masks. The resulting layer stack is a simple chain of matrix multiplications and bias additions, that can be contracted into a single weight matrix and bias vector. Inference then reduces to an affine transformation of the input sequence with these contracted parameters. We show that the method is akin to approximating the neural network with a first-order Taylor expansion around a dynamically updating reference point. The proposed algorithm is evaluated on a denoising convolutional autoencoder.
ISSN:2076-1465
DOI:10.23919/EUSIPCO.2019.8902997