Data-Aware Adaptive Pruning Model Compression Algorithm Based on a Group Attention Mechanism and Reinforcement Learning

The success of convolutional neural networks (CNNs) benefits from the stacking of convolutional layers, which improves the model's receptive field for image data but also causes a decrease in inference speed. To improve the inference speed of large convolutional network models without sacrifici...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE access Ročník 10; s. 82396 - 82406
Hlavní autoři: Yang, Zhi, Zhai, Yuan, Xiang, Yi, Wu, Jianquan, Shi, Jinliang, Wu, Ying
Médium: Journal Article
Jazyk:angličtina
Vydáno: Piscataway IEEE 2022
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Adaptive algorithms
Aircraft performance
Algorithms
Artificial neural networks
attention mechanism
Computational modeling
Convolutional neural networks
Datasets
Filtering algorithms
Inference
Machine learning
Model compression
pruning algorithm
Reinforcement learning
Training data
ISSN:2169-3536, 2169-3536
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:The success of convolutional neural networks (CNNs) benefits from the stacking of convolutional layers, which improves the model's receptive field for image data but also causes a decrease in inference speed. To improve the inference speed of large convolutional network models without sacrificing performance indicators too much, a data-aware adaptive pruning algorithm is proposed. The algorithm consists of two parts, namely, a channel pruning method based on the attention mechanism and a data-aware pruning policy based on reinforcement learning. Experimental results on the CIFAR-100 dataset show that the performance of the proposed pruning algorithm is reduced by only 2.05%, 1.93% and 5.66% after pruning the VGG19, ResNet56 and EfficientNet networks, respectively, but the speedup ratios are 3.63, 3.35, and 1.14, respectively, and the comprehensive pruning performance is the best. In addition, the generalization ability of the reconstruction model is evaluated on the ImageNet dataset and FGVC Aircraft dataset, and the performance of the proposed algorithm is the best, which shows that the proposed algorithm learns data-related information in the pruning process, that is, it is a data-aware algorithm.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2022.3188119