Deep neural networks compression learning based on multiobjective evolutionary algorithms

This work addresses the problem of deep neural network compression, which is a promising technique to reduce the number of network parameters and/or speed up the network evaluation process significantly. Most of the existing methods rely on domain experts’ experience for the selection of hyperparame...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Neurocomputing (Amsterdam) Ročník 378; s. 260 - 269
Hlavní autoři: Huang, Junhao, Sun, Weize, Huang, Lei
Médium: Journal Article
Jazyk:angličtina
Vydáno: Elsevier B.V 22.02.2020
Témata:
Evolutionary optimization
Expensive optimization
Neural network compression
Pareto-optimal solutions
Expensive optimization
Pareto-optimal solutions
Evolutionary optimization
Neural network compression
ISSN:0925-2312, 1872-8286
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:This work addresses the problem of deep neural network compression, which is a promising technique to reduce the number of network parameters and/or speed up the network evaluation process significantly. Most of the existing methods rely on domain experts’ experience for the selection of hyperparameters such as the rank and sparsity ratio of the weight matrix in order to get an appropriate compression result without serious performance downgrade. However, they usually suffer from heavy computational loads due to the large number of tests in revealing the best hyperparameters. In this work, we propose an efficient approach to network compression from the perspective of multiobjective evolution. The contributions in the paper are twofolds: (1) We build a multiobjective compression learning model that considers the model classification error rate and compression rate as two objectives in the optimization, which can provide a compromise of the tradeoffs between these two objectives. (2) A mechanism for approximate compressed model generation is devised in the framework of expensive multiobjective optimization, which is able to reduce the high model training costs involved in the optimization process. Experiments are carried out to confirm the superiority of the proposed algorithm.
ISSN:0925-2312
1872-8286
DOI:10.1016/j.neucom.2019.10.053