ResPrune: An energy-efficient restorative filter pruning method using stochastic optimization for accelerating CNN

Convolutional Neural Networks (CNNs) are frequently employed for image pattern recognition and other computer vision tasks. When over-parameterized deep learning models are used for inference, resource-constrained edge devices may struggle. As a result, model compression, particularly filter pruning...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Pattern recognition Jg. 155; S. 110671
Hauptverfasser: Jayasimhan, Anusha, P., Pabitha
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Elsevier Ltd 01.11.2024
Schlagworte:
Deep learning
Image classification
Model compression
Neural networks
Pruning
Deep learning
Pruning
Neural networks
Model compression
Image classification
ISSN:0031-3203, 1873-5142
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Convolutional Neural Networks (CNNs) are frequently employed for image pattern recognition and other computer vision tasks. When over-parameterized deep learning models are used for inference, resource-constrained edge devices may struggle. As a result, model compression, particularly filter pruning, has become critical. A reduction in model size might result in less calculation, resulting in faster hardware execution and lower energy consumption. One of the drawbacks of current pruning strategies is that once the filters are pruned, their weights are permanently lost. To address this constraint, we propose a unique two-phase pruning technique in which the filters to be pruned are selected using two criteria: l2-norm and redundancy. Second, rather than omitting the selected filters for all future epochs, we restore them to their original value with some stochasticity. Retaining the most optimal filter weights in earlier epochs enables the survival of the fittest filters, resulting in higher model convergence. Experiments on three benchmark datasets, CIFAR-10, CIFAR-100, and ILSVRC-2012, reveal that our strategy outperforms other state-of-the-art pruning methods by a minimum reduction of 57% FLOPs with an accuracy loss as minimal as 0.08 %. •Two-criteria (l2-norm and redundancy) imposed for pruning non-critical filters.•Stochastic optimization technique implemented for restoring most promising filters.•Proposed pruning method accelerates CNNs with more than 50% reduction in FLOPS.•Pruned models achieve nearly the same accuracy as baseline models.
ISSN:0031-3203
1873-5142
DOI:10.1016/j.patcog.2024.110671