A Unified DNN Weight Pruning Framework Using Reweighted Optimization Methods

To address the large model size and intensive computation requirement of deep neural networks (DNNs), weight pruning techniques have been proposed and generally fall into two categories, i.e., static regularization-based pruning and dynamic regularization-based pruning. However, the former method cu...

Full description

Saved in:
Bibliographic Details
Published in:2021 58th ACM/IEEE Design Automation Conference (DAC) pp. 493 - 498
Main Authors: Zhang, Tianyun, Ma, Xiaolong, Zhan, Zheng, Zhou, Shanglin, Ding, Caiwen, Fardad, Makan, Wang, Yanzhi
Format: Conference Proceeding
Language:English
Published: IEEE 05.12.2021
Subjects:
Computational modeling
Deep learning
Degradation
Design automation
Optimization methods
Training
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:To address the large model size and intensive computation requirement of deep neural networks (DNNs), weight pruning techniques have been proposed and generally fall into two categories, i.e., static regularization-based pruning and dynamic regularization-based pruning. However, the former method currently suffers either complex workloads or accuracy degradation, while the latter one takes a long time to tune the parameters to achieve the desired pruning rate without accuracy loss. In this paper, we propose a unified DNN weight pruning framework with dynamically updated regularization terms bounded by the designated constraint. Our proposed method increases the compression rate, reduces the training time and reduces the number of hyper-parameters compared with state-of-the-art ADMM-based hard constraint method.
DOI:10.1109/DAC18074.2021.9586152