Iterative Deep Neural Network Quantization With Lipschitz Constraint

Network quantization offers an effective solution to deep neural network compression for practical usage. Existing network quantization methods cannot theoretically guarantee the convergence. This paper proposes a novel iterative framework for network quantization with arbitrary bit-widths. We prese...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:IEEE transactions on multimedia Ročník 22; číslo 7; s. 1874 - 1888
Hlavní autori: Xu, Yuhui, Dai, Wenrui, Qi, Yingyong, Zou, Junni, Xiong, Hongkai
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Piscataway IEEE 01.07.2020
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:
Artificial neural networks
Computational modeling
Computer architecture
Convergence
Convolution
Image coding
iterative quantization
Lipschitz constraint
Measurement
Network compression
Neural networks
Object detection
Object recognition
Partitions
Performance enhancement
Quantization (signal)
Semantic segmentation
Semantics
State-of-the-art reviews
ISSN:1520-9210, 1941-0077
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Network quantization offers an effective solution to deep neural network compression for practical usage. Existing network quantization methods cannot theoretically guarantee the convergence. This paper proposes a novel iterative framework for network quantization with arbitrary bit-widths. We present two Lipschitz constraint based quantization strategies, namely width-level network quantization (WLQ) and multi-level network quantization (MLQ), for high-bit and extremely low-bit (ternary) quantization, respectively. In WLQ, Lipschitz based partition is developed to divide parameters in each layer into two groups: one for quantization and the other for re-training to eliminate the quantization loss. WLQ is further extended to MLQ by introducing layer partition to suppress the quantization loss for extremely low bit-widths. The Lipschitz based partition is proven to guarantee the convergence of the quantized networks. Moreover, the proposed framework is complementary to network compression methods such as activation quantization, pruning and efficient network architectures. The proposed framework is evaluated over extensive state-of-the-art deep neural networks, i.e., AlexNet, VGG-16, GoogleNet and ResNet18. Experimental results show that the proposed framework improves the performance of tasks like classification, object detection and semantic segmentation.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1520-9210
1941-0077
DOI:10.1109/TMM.2019.2949857