Binary neural networks: A survey

•We summarize the binary neural network methods and categorize them into the naive binarization and the optimized binarization.•The binary neural networks are mainly optimized using techniques including minimizing quantization error, improving the loss function, and reducing the gradient error.•We a...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Pattern recognition Jg. 105; S. 107281
Hauptverfasser: Qin, Haotong, Gong, Ruihao, Liu, Xianglong, Bai, Xiao, Song, Jingkuan, Sebe, Nicu
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Elsevier Ltd 01.09.2020
Schlagworte:
Binary neural network
Deep learning
Model acceleration
Model compression
Network quantization
Deep learning
Model acceleration
Binary neural network
Network quantization
Model compression
ISSN:0031-3203, 1873-5142
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:•We summarize the binary neural network methods and categorize them into the naive binarization and the optimized binarization.•The binary neural networks are mainly optimized using techniques including minimizing quantization error, improving the loss function, and reducing the gradient error.•We also discuss the hardware-friendly methods and the useful tricks of training binary neural networks.•We present the common datasets and network structures of evaluation, and compare the performance on different tasks.•We conclude and point out the future research trends. The binary neural network, largely saving the storage and computation, serves as a promising technique for deploying deep models on resource-limited devices. However, the binarization inevitably causes severe information loss, and even worse, its discontinuity brings difficulty to the optimization of the deep network. To address these issues, a variety of algorithms have been proposed, and achieved satisfying progress in recent years. In this paper, we present a comprehensive survey of these algorithms, mainly categorized into the native solutions directly conducting binarization, and the optimized ones using techniques like minimizing the quantization error, improving the network loss function, and reducing the gradient error. We also investigate other practical aspects of binary neural networks such as the hardware-friendly design and the training tricks. Then, we give the evaluation and discussions on different tasks, including image classification, object detection and semantic segmentation. Finally, the challenges that may be faced in future research are prospected.
ISSN:0031-3203
1873-5142
DOI:10.1016/j.patcog.2020.107281