A Unified DNN Weight Pruning Framework Using Reweighted Optimization Methods

To address the large model size and intensive computation requirement of deep neural networks (DNNs), weight pruning techniques have been proposed and generally fall into two categories, i.e., static regularization-based pruning and dynamic regularization-based pruning. However, the former method cu...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:2021 58th ACM/IEEE Design Automation Conference (DAC) S. 493 - 498
Hauptverfasser: Zhang, Tianyun, Ma, Xiaolong, Zhan, Zheng, Zhou, Shanglin, Ding, Caiwen, Fardad, Makan, Wang, Yanzhi
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 05.12.2021
Schlagworte:
Computational modeling
Deep learning
Degradation
Design automation
Optimization methods
Training
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:To address the large model size and intensive computation requirement of deep neural networks (DNNs), weight pruning techniques have been proposed and generally fall into two categories, i.e., static regularization-based pruning and dynamic regularization-based pruning. However, the former method currently suffers either complex workloads or accuracy degradation, while the latter one takes a long time to tune the parameters to achieve the desired pruning rate without accuracy loss. In this paper, we propose a unified DNN weight pruning framework with dynamically updated regularization terms bounded by the designated constraint. Our proposed method increases the compression rate, reduces the training time and reduces the number of hyper-parameters compared with state-of-the-art ADMM-based hard constraint method.
DOI:10.1109/DAC18074.2021.9586152