ResPrune: An energy-efficient restorative filter pruning method using stochastic optimization for accelerating CNN

Convolutional Neural Networks (CNNs) are frequently employed for image pattern recognition and other computer vision tasks. When over-parameterized deep learning models are used for inference, resource-constrained edge devices may struggle. As a result, model compression, particularly filter pruning...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Pattern recognition Ročník 155; s. 110671
Hlavní autoři: Jayasimhan, Anusha, P., Pabitha
Médium: Journal Article
Jazyk:angličtina
Vydáno: Elsevier Ltd 01.11.2024
Témata:
Deep learning
Image classification
Model compression
Neural networks
Pruning
Deep learning
Pruning
Neural networks
Model compression
Image classification
ISSN:0031-3203, 1873-5142
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Convolutional Neural Networks (CNNs) are frequently employed for image pattern recognition and other computer vision tasks. When over-parameterized deep learning models are used for inference, resource-constrained edge devices may struggle. As a result, model compression, particularly filter pruning, has become critical. A reduction in model size might result in less calculation, resulting in faster hardware execution and lower energy consumption. One of the drawbacks of current pruning strategies is that once the filters are pruned, their weights are permanently lost. To address this constraint, we propose a unique two-phase pruning technique in which the filters to be pruned are selected using two criteria: l2-norm and redundancy. Second, rather than omitting the selected filters for all future epochs, we restore them to their original value with some stochasticity. Retaining the most optimal filter weights in earlier epochs enables the survival of the fittest filters, resulting in higher model convergence. Experiments on three benchmark datasets, CIFAR-10, CIFAR-100, and ILSVRC-2012, reveal that our strategy outperforms other state-of-the-art pruning methods by a minimum reduction of 57% FLOPs with an accuracy loss as minimal as 0.08 %. •Two-criteria (l2-norm and redundancy) imposed for pruning non-critical filters.•Stochastic optimization technique implemented for restoring most promising filters.•Proposed pruning method accelerates CNNs with more than 50% reduction in FLOPS.•Pruned models achieve nearly the same accuracy as baseline models.
ISSN:0031-3203
1873-5142
DOI:10.1016/j.patcog.2024.110671