Data-Aware Adaptive Pruning Model Compression Algorithm Based on a Group Attention Mechanism and Reinforcement Learning
The success of convolutional neural networks (CNNs) benefits from the stacking of convolutional layers, which improves the model's receptive field for image data but also causes a decrease in inference speed. To improve the inference speed of large convolutional network models without sacrifici...
Uložené v:
| Vydané v: | IEEE access Ročník 10; s. 82396 - 82406 |
|---|---|
| Hlavní autori: | , , , , , |
| Médium: | Journal Article |
| Jazyk: | English |
| Vydavateľské údaje: |
Piscataway
IEEE
2022
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Predmet: | |
| ISSN: | 2169-3536, 2169-3536 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | The success of convolutional neural networks (CNNs) benefits from the stacking of convolutional layers, which improves the model's receptive field for image data but also causes a decrease in inference speed. To improve the inference speed of large convolutional network models without sacrificing performance indicators too much, a data-aware adaptive pruning algorithm is proposed. The algorithm consists of two parts, namely, a channel pruning method based on the attention mechanism and a data-aware pruning policy based on reinforcement learning. Experimental results on the CIFAR-100 dataset show that the performance of the proposed pruning algorithm is reduced by only 2.05%, 1.93% and 5.66% after pruning the VGG19, ResNet56 and EfficientNet networks, respectively, but the speedup ratios are 3.63, 3.35, and 1.14, respectively, and the comprehensive pruning performance is the best. In addition, the generalization ability of the reconstruction model is evaluated on the ImageNet dataset and FGVC Aircraft dataset, and the performance of the proposed algorithm is the best, which shows that the proposed algorithm learns data-related information in the pruning process, that is, it is a data-aware algorithm. |
|---|---|
| AbstractList | The success of convolutional neural networks (CNNs) benefits from the stacking of convolutional layers, which improves the model's receptive field for image data but also causes a decrease in inference speed. To improve the inference speed of large convolutional network models without sacrificing performance indicators too much, a data-aware adaptive pruning algorithm is proposed. The algorithm consists of two parts, namely, a channel pruning method based on the attention mechanism and a data-aware pruning policy based on reinforcement learning. Experimental results on the CIFAR-100 dataset show that the performance of the proposed pruning algorithm is reduced by only 2.05%, 1.93% and 5.66% after pruning the VGG19, ResNet56 and EfficientNet networks, respectively, but the speedup ratios are 3.63, 3.35, and 1.14, respectively, and the comprehensive pruning performance is the best. In addition, the generalization ability of the reconstruction model is evaluated on the ImageNet dataset and FGVC Aircraft dataset, and the performance of the proposed algorithm is the best, which shows that the proposed algorithm learns data-related information in the pruning process, that is, it is a data-aware algorithm. |
| Author | Shi, Jinliang Zhai, Yuan Wu, Ying Xiang, Yi Yang, Zhi Wu, Jianquan |
| Author_xml | – sequence: 1 givenname: Zhi orcidid: 0000-0003-1046-8654 surname: Yang fullname: Yang, Zhi organization: School of Intelligent Technology and Engineering, Chongqing University of Science and Technology, Chongqing, China – sequence: 2 givenname: Yuan surname: Zhai fullname: Zhai, Yuan organization: School of Intelligent Technology and Engineering, Chongqing University of Science and Technology, Chongqing, China – sequence: 3 givenname: Yi surname: Xiang fullname: Xiang, Yi organization: School of Intelligent Technology and Engineering, Chongqing University of Science and Technology, Chongqing, China – sequence: 4 givenname: Jianquan surname: Wu fullname: Wu, Jianquan organization: School of Intelligent Technology and Engineering, Chongqing University of Science and Technology, Chongqing, China – sequence: 5 givenname: Jinliang surname: Shi fullname: Shi, Jinliang organization: School of Intelligent Technology and Engineering, Chongqing University of Science and Technology, Chongqing, China – sequence: 6 givenname: Ying orcidid: 0000-0002-9683-5998 surname: Wu fullname: Wu, Ying email: wuying1992@cqust.edu.cn organization: School of Intelligent Technology and Engineering, Chongqing University of Science and Technology, Chongqing, China |
| BookMark | eNp9kU9v1DAQxSNUJErpJ-jFEucs_hMnzjGEUiptBaJwtsb2ZOtVEgfHS8W3JyEFIQ74Yut53m9G815mZ2MYMcuuGN0xRus3Tdte39_vOOV8J5hSjNXPsnPOyjoXUpRnf71fZJfzfKTLUYskq_Ps8R0kyJtHiEgaB1Py35F8iqfRjwdyFxz2pA3DFHGefRhJ0x9C9OlhIG9hRkcWCchNDKeJNCnhmNaiO7QPMPp5IDA68hn92IVocVi-yR4hruxX2fMO-hkvn-6L7Ov76y_th3z_8ea2bfa5LahKOROutka5mltjpeuoYA4Y5UqqCiXw0kiQYAxYwUpRUqgclIZKZbpKyLoUF9ntxnUBjnqKfoD4Qwfw-pcQ4kFDTN72qGXBrVv25yy1RYUKys4ZXMjCmIqDWFivN9YUw7cTzkkfwymOy_iaV1QUQhVy7VhvVTaGeY7YaesTrItJEXyvGdVrbHqLTa-x6afYFq_4x_t74v-7rjaXR8Q_jloxURVM_ATj2KbX |
| CODEN | IAECCG |
| CitedBy_id | crossref_primary_10_1109_ACCESS_2024_3457863 crossref_primary_10_1109_ACCESS_2025_3533419 crossref_primary_10_1109_ACCESS_2022_3229192 crossref_primary_10_1109_ACCESS_2024_3498904 |
| Cites_doi | 10.1109/CVPR.2016.280 10.1007/s11263-015-0816-y 10.1109/ICCV.2017.298 10.3390/electronics9081209 10.1007/s11042-017-4440-4 10.1109/ICCV.2017.541 10.1109/CVPR.2016.90 10.1109/ICCV.2017.155 10.1109/CVPR42600.2020.00160 10.1109/ICME.2017.8019465 10.1109/CVPR.2019.00290 10.1609/aaai.v33i01.33015676 10.1109/CVPR.2018.00890 10.3390/app10196866 10.1111/mice.12449 10.1109/MSP.2017.2765695 10.3389/fnins.2019.00405 10.1109/CVPR.2019.01152 10.1016/j.imavis.2021.104143 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022 |
| DBID | 97E ESBDL RIA RIE AAYXX CITATION 7SC 7SP 7SR 8BQ 8FD JG9 JQ2 L7M L~C L~D DOA |
| DOI | 10.1109/ACCESS.2022.3188119 |
| DatabaseName | IEEE Xplore (IEEE) IEEE Xplore Open Access Journals IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Engineered Materials Abstracts METADEX Technology Research Database Materials Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional DOAJ Directory of Open Access Journals |
| DatabaseTitle | CrossRef Materials Research Database Engineered Materials Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace METADEX Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Materials Research Database |
| Database_xml | – sequence: 1 dbid: DOA name: DOAJ Directory of Open Access Journals url: https://www.doaj.org/ sourceTypes: Open Website – sequence: 2 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 2169-3536 |
| EndPage | 82406 |
| ExternalDocumentID | oai_doaj_org_article_542cd188dc0c47e8a6fdbeac33bb72a3 10_1109_ACCESS_2022_3188119 9813741 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Chongqing Talent Plan, Cooperation Project between Chongqing Municipal Undergraduate Universities and Institutes affiliated to the Chinese Academy of Sciences, in 2021 grantid: HZ2021015 funderid: 10.13039/501100002367 – fundername: National Natural Science and Foundation of China grantid: 61903055 funderid: 10.13039/501100001809 |
| GroupedDBID | 0R~ 4.4 5VS 6IK 97E AAJGR ABAZT ABVLG ACGFS ADBBV AGSQL ALMA_UNASSIGNED_HOLDINGS BCNDV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD ESBDL GROUPED_DOAJ IPLJI JAVBF KQ8 M43 M~E O9- OCL OK1 RIA RIE RNS AAYXX CITATION 7SC 7SP 7SR 8BQ 8FD JG9 JQ2 L7M L~C L~D |
| ID | FETCH-LOGICAL-c408t-13d9cb8d92cbc5df031da1028587e5a26b5a5abbac316360a7da6b058bf735963 |
| IEDL.DBID | DOA |
| ISICitedReferencesCount | 6 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000841773400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2169-3536 |
| IngestDate | Tue Oct 14 15:00:10 EDT 2025 Mon Jun 30 04:53:45 EDT 2025 Sat Nov 29 06:32:14 EST 2025 Tue Nov 18 22:19:02 EST 2025 Wed Aug 27 02:14:03 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Language | English |
| License | https://creativecommons.org/licenses/by/4.0/legalcode |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c408t-13d9cb8d92cbc5df031da1028587e5a26b5a5abbac316360a7da6b058bf735963 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0003-1046-8654 0000-0002-9683-5998 |
| OpenAccessLink | https://doaj.org/article/542cd188dc0c47e8a6fdbeac33bb72a3 |
| PQID | 2703438456 |
| PQPubID | 4845423 |
| PageCount | 11 |
| ParticipantIDs | ieee_primary_9813741 crossref_primary_10_1109_ACCESS_2022_3188119 crossref_citationtrail_10_1109_ACCESS_2022_3188119 doaj_primary_oai_doaj_org_article_542cd188dc0c47e8a6fdbeac33bb72a3 proquest_journals_2703438456 |
| PublicationCentury | 2000 |
| PublicationDate | 20220000 2022-00-00 20220101 2022-01-01 |
| PublicationDateYYYYMMDD | 2022-01-01 |
| PublicationDate_xml | – year: 2022 text: 20220000 |
| PublicationDecade | 2020 |
| PublicationPlace | Piscataway |
| PublicationPlace_xml | – name: Piscataway |
| PublicationTitle | IEEE access |
| PublicationTitleAbbrev | Access |
| PublicationYear | 2022 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref13 ref15 ref14 Maji (ref22) 2013 ref11 ref10 ref2 ref1 ref16 ref19 ref18 Li (ref8) Peng (ref17) ref24 ref20 ref21 Srivastava (ref26) 2014; 15 Meng (ref12) 2020 ref27 ref7 ref9 ref3 ref6 ref5 Tan (ref25) Simonyan (ref23) Han (ref4) |
| References_xml | – ident: ref9 doi: 10.1109/CVPR.2016.280 – ident: ref21 doi: 10.1007/s11263-015-0816-y – ident: ref16 doi: 10.1109/ICCV.2017.298 – ident: ref19 doi: 10.3390/electronics9081209 – ident: ref24 doi: 10.1007/s11042-017-4440-4 – ident: ref15 doi: 10.1109/ICCV.2017.541 – volume-title: arXiv:1306.5151 year: 2013 ident: ref22 article-title: Fine-grained visual classification of aircraft – ident: ref20 doi: 10.1109/CVPR.2016.90 – ident: ref10 doi: 10.1109/ICCV.2017.155 – ident: ref13 doi: 10.1109/CVPR42600.2020.00160 – ident: ref1 doi: 10.1109/ICME.2017.8019465 – start-page: 5113 volume-title: Proc. Int. Conf. Mach. Learn. ident: ref17 article-title: Collaborative channel pruning for deep networks – start-page: 1135 volume-title: Proc. Adv. Neural Inf. Process. Syst. ident: ref4 article-title: Learning both weights and connections for efficient neural network – ident: ref11 doi: 10.1109/CVPR.2019.00290 – start-page: 1 volume-title: Proc. Int. Conf. Learn. Represent. ident: ref8 article-title: Pruning filters for efficient ConvNets – ident: ref6 doi: 10.1609/aaai.v33i01.33015676 – ident: ref3 doi: 10.1109/CVPR.2018.00890 – start-page: 6105 volume-title: Proc. Int. Conf. Mach. Learn. ident: ref25 article-title: EfficientNet: Rethinking model scaling for convolutional neural networks – ident: ref2 doi: 10.3390/app10196866 – ident: ref7 doi: 10.1111/mice.12449 – ident: ref27 doi: 10.1109/MSP.2017.2765695 – volume-title: arXiv:2009.14410 year: 2020 ident: ref12 article-title: Pruning filter in filter – ident: ref18 doi: 10.3389/fnins.2019.00405 – volume: 15 start-page: 1929 issue: 1 year: 2014 ident: ref26 article-title: Dropout: A simple way to prevent neural networks from overfitting publication-title: J. Mach. Learn. Res. – ident: ref5 doi: 10.1109/CVPR.2019.01152 – start-page: 1 volume-title: Proc. Int. Conf. Learn. Represent. ident: ref23 article-title: Very deep convolutional networks for large-scale image recognition – ident: ref14 doi: 10.1016/j.imavis.2021.104143 |
| SSID | ssj0000816957 |
| Score | 2.2755334 |
| Snippet | The success of convolutional neural networks (CNNs) benefits from the stacking of convolutional layers, which improves the model's receptive field for image... The success of convolutional neural networks (CNNs) benefits from the stacking of convolutional layers, which improves the model’s receptive field for image... |
| SourceID | doaj proquest crossref ieee |
| SourceType | Open Website Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 82396 |
| SubjectTerms | Adaptive algorithms Aircraft performance Algorithms Artificial neural networks attention mechanism Computational modeling Convolutional neural networks Datasets Filtering algorithms Inference Machine learning Model compression pruning algorithm Reinforcement learning Training data |
| SummonAdditionalLinks | – databaseName: IEEE Electronic Library (IEL) dbid: RIE link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1Lb9QwELZKxQEOvApi24J84NjQxI_YOaYLFReqCoHUmzV-pKy0zVbZLP37eBw3qgRC4hZZduT48zw88XxDyIfGlVH1Rs-tKr0vhCh9AUKGouQqqK6uO1F3qdiEurjQV1fN5R45mXNhQgjp8ln4iI_pX77fuB2Gyk4bXXGFWeqPlKqnXK05noIFJBqpMrFQVTan7XIZvyEeARmLJ1OtK2TTeWB8Ekd_LqryhyZO5uX8-f9N7AV5lt1I2k64vyR7oX9Fnj4gFzwgd59ghKK9gyHQ1sMt6jV6OewwEEKxBNqaojKY7sH2tF1fb4bV-POGnkXD5mlsApoiU7Qdx-lSJP0aMFF4tb2h0Hv6LSTaVZcijDQztV6_Jj_OP39ffilymYXCiVJjMXrfOKt9w5x10ndRzD2g3yG1ChJYbSVIsBYcr5BdDJSH2pZS205xGQX4DdnvN314Syh0inXRBYmLEQTUla2CY_EIB54rKyQsCLtff-MyBzmWwlibdBYpGzOBZhA0k0FbkJN50O1EwfHv7mcI7NwV-bNTQ0TMZHE0UjDnY38ft6pQQUPdeRttEOfWKgZ8QQ4Q5fklGeAFOb7fJibL-tawqDQF19ETPfz7qCPyBCc4BW6Oyf447MI78tj9Glfb4X3axr8BtILx0Q priority: 102 providerName: IEEE |
| Title | Data-Aware Adaptive Pruning Model Compression Algorithm Based on a Group Attention Mechanism and Reinforcement Learning |
| URI | https://ieeexplore.ieee.org/document/9813741 https://www.proquest.com/docview/2703438456 https://doaj.org/article/542cd188dc0c47e8a6fdbeac33bb72a3 |
| Volume | 10 |
| WOSCitedRecordID | wos000841773400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAON databaseName: DOAJ Directory of Open Access Journals customDbUrl: eissn: 2169-3536 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000816957 issn: 2169-3536 databaseCode: DOA dateStart: 20130101 isFulltext: true titleUrlDefault: https://www.doaj.org/ providerName: Directory of Open Access Journals – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2169-3536 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000816957 issn: 2169-3536 databaseCode: M~E dateStart: 20130101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1Lb9QwELZQxQEOCCiILaXygSNRY8eO7WO6tOLSqkIg9WaNHykrbdMqm9Ibv52x465WQoILlxwsx4k9D8-M7O8j5KPxNbpejNxYHUIlRB0qEDJWdaOi6tu2F22fySbUxYW-ujKXO1Rf6UzYDA88L9yxFNwHpnXAQYWKGto-OPQWTeOc4pBxPmtldpKp7IM1a41UBWaI1ea4Wy5xRpgQco55qtYsYevsbEUZsb9QrPzhl_Nmc_aSvChRIu3mv3tFnsThNXm-gx24Tx4-wwRV9wBjpF2Au-S26OV4n-ocNDGcrWmy9fmY60C79fXtuJp-3NAT3LcCxSagufBEu2mazzzS85juAa82NxSGQL_GjKrqcwGRFiDW6zfk-9npt-WXqrAoVF7UOnHNB-OdDoZ752Xo0YoDpLBCahUl8NZJkOAcLilL4GGgArSultr1qpFon2_J3nA7xHeEQq94jxFGo0QU0DLHoueYoUFolBMSFoQ_Lqj1BWI8MV2sbU41amNnKdgkBVuksCCfti_dzQgbf-9-kiS17ZrgsXMDKo0tSmP_pTQLsp_kvB3EaIZzYgty-Ch3W0x5Yzn6RNFoDDQP_sen35NnaTpzFeeQ7E3jffxAnvqf02ozHmUtxuf5r9OjfBfxNy3y-HA |
| linkProvider | Directory of Open Access Journals |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1Lb9QwELaqggQceBXElgI-cGxo4tixc0wXqiLaVYWK1Js1fqRdaZutdrP079fjuFElEBK3yLIjx5_n4YnnG0I-1zYPqjd4bkXuXMZ57jLgwmd5Kb1sq6rlVRuLTcjZTF1c1GdbZH_MhfHex8tn_gs-xn_5bmk3GCo7qFVRSsxSf4SVs1K21hhRwRIStZCJWqjI64NmOg1fEQ6BjIWzqVIF8uk8MD-RpT-VVflDF0cDc_Ti_6b2kjxPjiRtBuRfkS3fvSbPHtAL7pDbr9BD1tzCytPGwQ1qNnq22mAohGIRtAVFdTDchO1os7hcrub91TU9DKbN0dAENMamaNP3w7VIeuoxVXi-vqbQOfrTR-JVG2OMNHG1Xr4hv46-nU-Ps1RoIbM8V1iO3tXWKFcza6xwbRB0B-h5CCW9AFYZAQKMAVsWyC8G0kFlcqFMK0sRRPgt2e6WnX9HKLSStcEJCYvhOVSFKbxl4RAHrpSGC5gQdr_-2iYWciyGsdDxNJLXegBNI2g6gTYh--Ogm4GE49_dDxHYsSsyaMeGgJhOAqkFZ9aF_i5sVi69gqp1JlihsjRGMignZAdRHl-SAJ6QvfttopO0rzULapOXKviiu38f9Yk8OT4_PdEn32c_3pOnONkhjLNHtvvVxn8gj-3vfr5efYxb-g5t9fUa |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Data-Aware+Adaptive+Pruning+Model+Compression+Algorithm+Based+on+a+Group+Attention+Mechanism+and+Reinforcement+Learning&rft.jtitle=IEEE+access&rft.au=Yang%2C+Zhi&rft.au=Zhai%2C+Yuan&rft.au=Xiang%2C+Yi&rft.au=Wu%2C+Jianquan&rft.date=2022&rft.pub=IEEE&rft.eissn=2169-3536&rft.volume=10&rft.spage=82396&rft.epage=82406&rft_id=info:doi/10.1109%2FACCESS.2022.3188119&rft.externalDocID=9813741 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2169-3536&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2169-3536&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2169-3536&client=summon |