Linear Approximation of Deep Neural Networks for Efficient Inference on Video Data

Sequential data such as video are characterized by spatio-temporal correlations. As of yet, few deep learning algorithms exploit them to decrease the often massive cost during inference. This work leverages correlations in video data to linearize part of a deep neural network and thus reduce its siz...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:2019 27th European Signal Processing Conference (EUSIPCO) s. 1 - 5
Hlavní autoři: Rueckauer, Bodo, Liu, Shih-Chii
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: EURASIP 01.09.2019
Témata:
Biological neural networks
compression
Convolution
Correlation
Deep neural networks
linearization
Neurons
Noise reduction
sequential data
Task analysis
Taylor series
video
ISSN:2076-1465
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Sequential data such as video are characterized by spatio-temporal correlations. As of yet, few deep learning algorithms exploit them to decrease the often massive cost during inference. This work leverages correlations in video data to linearize part of a deep neural network and thus reduce its size and computational cost. Drawing upon the simplicity of the typically used rectifier activation function, we replace the ReLU function by dynamically updating masks. The resulting layer stack is a simple chain of matrix multiplications and bias additions, that can be contracted into a single weight matrix and bias vector. Inference then reduces to an affine transformation of the input sequence with these contracted parameters. We show that the method is akin to approximating the neural network with a first-order Taylor expansion around a dynamically updating reference point. The proposed algorithm is evaluated on a denoising convolutional autoencoder.
ISSN:2076-1465
DOI:10.23919/EUSIPCO.2019.8902997