DeepCABAC: A Universal Compression Algorithm for Deep Neural Networks

In the past decade deep neural networks (DNNs) have shown state-of-the-art performance on a wide range of complex machine learning tasks. Many of these results have been achieved while growing the size of DNNs, creating a demand for efficient compression and transmission of them. In this work we pre...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE journal of selected topics in signal processing Ročník 14; číslo 4; s. 700 - 714
Hlavní autoři: Wiedemann, Simon, Kirchhoffer, Heiner, Matlage, Stefan, Haase, Paul, Marban, Arturo, Marinc, Talmaj, Neumann, David, Nguyen, Tung, Schwarz, Heiko, Wiegand, Thomas, Marpe, Detlev, Samek, Wojciech
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York IEEE 01.05.2020
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Algorithms
arithmetic coding
Artificial neural networks
Coding
Coding standards
Cognitive tasks
Compression algorithms
Decoding
Deep learning
efficient representation
Machine learning
Measurement
neural network compression
Neural networks
Quantization (signal)
rate-distortion quantization
Source code
Source coding
Task complexity
Video compression
ISSN:1932-4553, 1941-0484
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:In the past decade deep neural networks (DNNs) have shown state-of-the-art performance on a wide range of complex machine learning tasks. Many of these results have been achieved while growing the size of DNNs, creating a demand for efficient compression and transmission of them. In this work we present DeepCABAC, a universal compression algorithm for DNNs that is based on applying Context-based Adaptive Binary Arithmetic Coder (CABAC) to the DNN parameters. CABAC was originally designed for the H.264/AVC video coding standard and became the state-of-the-art for the lossless compression part of video compression. DeepCABAC applies a novel quantization scheme that minimizes a rate-distortion function while simultaneously taking the impact of quantization to the DNN performance into account. Experimental results show that DeepCABAC consistently attains higher compression rates than previously proposed coding techniques for DNN compression. For instance, it is able to compress the VGG16 ImageNet model by x63.6 with no loss of accuracy, thus being able to represent the entire network with merely 9 MB. The source code for encoding and decoding can be found at https://github.com/fraunhoferhhi/DeepCABAC .
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1932-4553
1941-0484
DOI:10.1109/JSTSP.2020.2969554