Adaptive Scaling Filter Pruning Method for Vision Networks With Embedded Devices
Owing to improvements in computing power, deep learning technology using convolutional neural networks (CNNs) has recently been used in various fields. However, using CNNs on edge devices is challenging because of the large computation required to achieve high performance. To solve this problem, pru...
Gespeichert in:
| Veröffentlicht in: | IEEE access Jg. 12; S. 123771 - 123781 |
|---|---|
| Hauptverfasser: | , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
Piscataway
IEEE
2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Schlagworte: | |
| ISSN: | 2169-3536, 2169-3536 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | Owing to improvements in computing power, deep learning technology using convolutional neural networks (CNNs) has recently been used in various fields. However, using CNNs on edge devices is challenging because of the large computation required to achieve high performance. To solve this problem, pruning, which reduces redundant parameters and computations, has been widely studied. However, a conventional pruning method requires two learning processes, which are time-consuming and resource-intensive, and it is difficult to reflect the redundancy in the pruned network because it only performs pruning once on the unpruned network. Therefore, in this paper, we utilize a single learning process and propose an adaptive scaling method that dynamically adjusts the size of the network to reflect the changing redundancy in the pruned network. To verify the performance of each method, we compare the performance of the proposed methods by conducting experiments on various datasets and networks. In our experiments using the ImageNet dataset on ResNet-50, pruning FLOPs by 50.1% and 74.0% resulted in a decrease in top-1 accuracy by 0.92% and 3.38%, respectively, and improved inference time by 26.4% and 58.9%, respectively. In addition, pruning FLOPs by 27.37%, 36.84% and 46.41% using the COCO dataset on YOLOv7, reduced mAP(0.5-0.95) by 1.2%, 2.2% and 2.9%, respectively, and improved inference time by 12.9%, 16.9% and19.3%. |
|---|---|
| AbstractList | Owing to improvements in computing power, deep learning technology using convolutional neural networks (CNNs) has recently been used in various fields. However, using CNNs on edge devices is challenging because of the large computation required to achieve high performance. To solve this problem, pruning, which reduces redundant parameters and computations, has been widely studied. However, a conventional pruning method requires two learning processes, which are time-consuming and resource-intensive, and it is difficult to reflect the redundancy in the pruned network because it only performs pruning once on the unpruned network. Therefore, in this paper, we utilize a single learning process and propose an adaptive scaling method that dynamically adjusts the size of the network to reflect the changing redundancy in the pruned network. To verify the performance of each method, we compare the performance of the proposed methods by conducting experiments on various datasets and networks. In our experiments using the ImageNet dataset on ResNet-50, pruning FLOPs by 50.1% and 74.0% resulted in a decrease in top-1 accuracy by 0.92% and 3.38%, respectively, and improved inference time by 26.4% and 58.9%, respectively. In addition, pruning FLOPs by 27.37%, 36.84% and 46.41% using the COCO dataset on YOLOv7, reduced mAP(0.5-0.95) by 1.2%, 2.2% and 2.9%, respectively, and improved inference time by 12.9%, 16.9% and19.3%. |
| Author | Kim, Yongwoo Ko, Hyunjun Kang, Jin-Ku |
| Author_xml | – sequence: 1 givenname: Hyunjun orcidid: 0009-0005-8955-744X surname: Ko fullname: Ko, Hyunjun organization: Department of Electrical and Computer Engineering, Inha University, Incheon, Republic of Korea – sequence: 2 givenname: Jin-Ku orcidid: 0000-0002-3752-3740 surname: Kang fullname: Kang, Jin-Ku organization: Department of Electrical and Computer Engineering, Inha University, Incheon, Republic of Korea – sequence: 3 givenname: Yongwoo orcidid: 0000-0002-1011-2319 surname: Kim fullname: Kim, Yongwoo email: yongwoo.kim@knue.ac.kr organization: Department of Technology Education, Korea National University of Education, Cheongju, Republic of Korea |
| BookMark | eNpNUctKQzEQDaKgVr9AFwHXrXnfZllqfYAvqI9lSJOJptabmtwq_r23XpHOZmYOc84ZOPtou041IHREyYBSok9H4_FkOh0wwsSACyk401toj1Gl-1xytb0x76LDUuakrWELyWoP3Y-8XTbxE_DU2UWsX_B5XDSQ8X1e1ev1BprX5HFIGT_FElONb6H5Svmt4OfYvOLJ-wy8B4_P4DM6KAdoJ9hFgcO_3kOP55OH8WX_-u7iajy67jsuddMPzldKO6qEp7YSVmsWgtQgJBGBOgnEWcusm2kbqgoCU0IxClwNGTAiOO-hq07XJzs3yxzfbf42yUbzC6T8YmxuoluAUdwFC8xXknERwFnpvVJhyKsZ8UKLVuuk01rm9LGC0ph5WuW6fd9wSphWWvG1I--uXE6lZAj_rpSYdRKmS8KskzB_SbSs444VAWCDoZSQquI_3kGF7g |
| CODEN | IAECCG |
| Cites_doi | 10.1109/jstsp.2024.3387299 10.1109/ICCV.2017.298 10.1007/978-3-030-58536-5_38 10.1007/s11263-014-0733-5 10.1145/3065386 10.1109/5.726791 10.1109/CVPR.2015.7298594 10.1109/TPAMI.2016.2577031 10.1109/CVPR.2018.00958 10.1007/978-3-319-24574-4_28 10.1109/CVPR42600.2020.00160 10.1007/978-3-030-01270-0_19 10.1109/ICCV48922.2021.00447 10.1016/j.patcog.2020.107461 10.1109/ACCESS.2022.3188323 10.1109/CVPR.2009.5206848 10.48550/arXiv.1802.02611 10.1109/CVPR52729.2023.01544 10.1109/CVPR.2016.90 10.5555/3045118.3045167 10.1109/CVPR.2015.7298965 10.l007/978-3-319-46448-0_2 10.1109/CVPR52729.2023.00721 10.1109/CVPR.2019.01152 10.1109/CVPR52688.2022.01197 10.1109/CVPR.2019.00290 10.1109/VTC2023-Spring57618.2023.10200157 10.1007/978-3-319-10602-1_48 10.24963/ijcai.2018/309 10.1109/CVPR52688.2022.00029 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024 |
| DBID | 97E ESBDL RIA RIE AAYXX CITATION 7SC 7SP 7SR 8BQ 8FD JG9 JQ2 L7M L~C L~D DOA |
| DOI | 10.1109/ACCESS.2024.3454329 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005-present IEEE Open Access Journals IEEE All-Society Periodicals Package (ASPP) 1998-Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Engineered Materials Abstracts METADEX Technology Research Database Materials Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional DOAJ Directory of Open Access Journals |
| DatabaseTitle | CrossRef Materials Research Database Engineered Materials Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace METADEX Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Materials Research Database |
| Database_xml | – sequence: 1 dbid: DOA name: DOAJ Directory of Open Access Journals url: https://www.doaj.org/ sourceTypes: Open Website – sequence: 2 dbid: RIE name: IEEE/IET Electronic Library url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 2169-3536 |
| EndPage | 123781 |
| ExternalDocumentID | oai_doaj_org_article_63cfae2d75234feca5dd66f837b0d494 10_1109_ACCESS_2024_3454329 10664567 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: MSIT through the Information Technology Research Center (ITRC) Support Program supervised by the Institute for Information and Communications Technology Planning and Evaluation (IITP) grantid: IITP-2021-0-02052 – fundername: Korean Government through the Ministry of Science and ICT (MSIT), South Korea grantid: 2022R1G1A1007415 – fundername: National Research Foundation of Korea (NRF) Grant |
| GroupedDBID | 0R~ 4.4 5VS 6IK 97E AAJGR ABAZT ABVLG ACGFS ADBBV AGSQL ALMA_UNASSIGNED_HOLDINGS BCNDV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD ESBDL GROUPED_DOAJ IPLJI JAVBF KQ8 M43 M~E O9- OCL OK1 RIA RIE RNS AAYXX CITATION 7SC 7SP 7SR 8BQ 8FD JG9 JQ2 L7M L~C L~D |
| ID | FETCH-LOGICAL-c359t-fcd769c164d1a74a992ff59e4504f1c5e0caa2acb9af77ef264621e3682e20433 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 0 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001310497000001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2169-3536 |
| IngestDate | Fri Oct 03 12:42:22 EDT 2025 Mon Jun 30 16:34:05 EDT 2025 Sat Nov 29 04:27:04 EST 2025 Wed Aug 27 02:01:30 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Language | English |
| License | https://creativecommons.org/licenses/by-nc-nd/4.0 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c359t-fcd769c164d1a74a992ff59e4504f1c5e0caa2acb9af77ef264621e3682e20433 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0002-3752-3740 0000-0002-1011-2319 0009-0005-8955-744X |
| OpenAccessLink | https://ieeexplore.ieee.org/document/10664567 |
| PQID | 3102969633 |
| PQPubID | 4845423 |
| PageCount | 11 |
| ParticipantIDs | doaj_primary_oai_doaj_org_article_63cfae2d75234feca5dd66f837b0d494 crossref_primary_10_1109_ACCESS_2024_3454329 ieee_primary_10664567 proquest_journals_3102969633 |
| PublicationCentury | 2000 |
| PublicationDate | 20240000 2024-00-00 20240101 2024-01-01 |
| PublicationDateYYYYMMDD | 2024-01-01 |
| PublicationDate_xml | – year: 2024 text: 20240000 |
| PublicationDecade | 2020 |
| PublicationPlace | Piscataway |
| PublicationPlace_xml | – name: Piscataway |
| PublicationTitle | IEEE access |
| PublicationTitleAbbrev | Access |
| PublicationYear | 2024 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref13 Kang (ref30) ref12 ref15 ref37 ref36 ref31 ref11 ref33 ref10 ref32 Krizhevsky (ref35) 2009 ref2 ref1 ref39 ref16 ref38 ref19 ref18 Wang (ref34) 2022 Simonyan (ref4) 2014 Li (ref27) 2016 Nonnenmacher (ref29) 2021 Xu (ref14) 2018 ref24 ref23 ref26 ref25 ref20 ref22 ref21 (ref40) 2022 Chen (ref17); 34 ref8 ref7 ref9 ref3 ref6 ref5 Sui (ref28); 34 |
| References_xml | – year: 2018 ident: ref14 article-title: Hybrid pruning: Thinner sparse networks for fast inference on edge devices publication-title: arXiv:1811.00482 – ident: ref1 doi: 10.1109/jstsp.2024.3387299 – ident: ref19 doi: 10.1109/ICCV.2017.298 – ident: ref18 doi: 10.1007/978-3-030-58536-5_38 – year: 2022 ident: ref34 article-title: Trainability preserving neural pruning publication-title: arXiv:2207.12534 – ident: ref38 doi: 10.1007/s11263-014-0733-5 – ident: ref6 doi: 10.1145/3065386 – ident: ref7 doi: 10.1109/5.726791 – ident: ref5 doi: 10.1109/CVPR.2015.7298594 – volume: 34 start-page: 19637 volume-title: Proc. Adv. Neural Inf. Process. Syst. ident: ref17 article-title: Only train once: A one-shot neural network training and pruning framework – year: 2021 ident: ref29 article-title: SOSP: Efficiently capturing global correlations by second-order structured pruning publication-title: arXiv:2110.11395 – start-page: 5122 volume-title: Proc. Int. Conf. Mach. Learn. ident: ref30 article-title: Operation-aware soft channel pruning using differentiable masks – ident: ref10 doi: 10.1109/TPAMI.2016.2577031 – ident: ref20 doi: 10.1109/CVPR.2018.00958 – ident: ref11 doi: 10.1007/978-3-319-24574-4_28 – ident: ref25 doi: 10.1109/CVPR42600.2020.00160 – ident: ref22 doi: 10.1007/978-3-030-01270-0_19 – ident: ref33 doi: 10.1109/ICCV48922.2021.00447 – ident: ref23 doi: 10.1016/j.patcog.2020.107461 – ident: ref15 doi: 10.1109/ACCESS.2022.3188323 – ident: ref36 doi: 10.1109/CVPR.2009.5206848 – ident: ref12 doi: 10.48550/arXiv.1802.02611 – ident: ref31 doi: 10.1109/CVPR52729.2023.01544 – ident: ref3 doi: 10.1109/CVPR.2016.90 – ident: ref39 doi: 10.5555/3045118.3045167 – ident: ref13 doi: 10.1109/CVPR.2015.7298965 – volume: 34 start-page: 24604 volume-title: Proc. Adv. Neural Inf. Process. Syst. (NIPS) ident: ref28 article-title: Chip: Channel independence-based pruning for compact neural networks – ident: ref9 doi: 10.l007/978-3-319-46448-0_2 – ident: ref8 doi: 10.1109/CVPR52729.2023.00721 – volume-title: Filter-Gap year: 2022 ident: ref40 – ident: ref24 doi: 10.1109/CVPR.2019.01152 – ident: ref16 doi: 10.1109/CVPR52688.2022.01197 – ident: ref21 doi: 10.1109/CVPR.2019.00290 – year: 2016 ident: ref27 article-title: Pruning filters for efficient convnets publication-title: arXiv:1608.08710 – ident: ref2 doi: 10.1109/VTC2023-Spring57618.2023.10200157 – volume-title: Learning multiple layers of features from tiny images year: 2009 ident: ref35 – year: 2014 ident: ref4 article-title: Very deep convolutional networks for large-scale image recognition publication-title: arXiv:1409.1556 – ident: ref37 doi: 10.1007/978-3-319-10602-1_48 – ident: ref26 doi: 10.24963/ijcai.2018/309 – ident: ref32 doi: 10.1109/CVPR52688.2022.00029 |
| SSID | ssj0000816957 |
| Score | 2.2998722 |
| Snippet | Owing to improvements in computing power, deep learning technology using convolutional neural networks (CNNs) has recently been used in various fields.... |
| SourceID | doaj proquest crossref ieee |
| SourceType | Open Website Aggregation Database Index Database Publisher |
| StartPage | 123771 |
| SubjectTerms | Adaptive filters Adaptive systems Artificial neural networks Batch normalization Computer vision convolutional neural network Convolutional neural networks Datasets Deep learning Filtering algorithms Inference inference time Information filters Machine learning network compression Pruning Quantization (signal) Redundancy Training |
| SummonAdditionalLinks | – databaseName: DOAJ Directory of Open Access Journals dbid: DOA link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV27TsMwFLUQYoAB8SiiUJAHRkITJ7bjsZRWDFBVAko3y_FDdKBUbcr3c-2kEImBhdWK5Pjc5PocP85F6KqglCtGbcTCNiNQgEi5nEUwFREjjLIqVFGYPPDRKJ9OxbhR6sufCavsgSvguizVTlliOCimzFmtqDGMOdBVRWwyEZxAYy4aYirk4DxhgvLaZiiJRbfX78OIQBCS7CbNaJYGUvkzFQXH_rrEyq-8HCab4QHar1ki7lVvd4i27PwI7TW8A4_RuGfUwucq_AQwQxMezvzONx4v136tAz-G2tAYSCmehAvkeFQd-V7h11n5hgfvhYWsY_CdDdmihV6Gg-f-fVSXR4h0SkUZOW04Exr0jkkUz5QQxDkqLACeuURTG2uliNKFUI5z64D6MJLYlOXE-hux6Qnann_M7SnCjBU2psrZXAA9MqoQ3vOlIAVw70zzuI2uN0jJReWCIYN6iIWsgJUeWFkD20a3Hs3vR72FdWiAwMo6sPKvwLZRy8ei0R9jwPZ4G3U2wZH1_7aSQFKJ9_lJ07P_6Psc7frxVEstHbRdLtf2Au3oz3K2Wl6GT-0L3fXV0g priority: 102 providerName: Directory of Open Access Journals |
| Title | Adaptive Scaling Filter Pruning Method for Vision Networks With Embedded Devices |
| URI | https://ieeexplore.ieee.org/document/10664567 https://www.proquest.com/docview/3102969633 https://doaj.org/article/63cfae2d75234feca5dd66f837b0d494 |
| Volume | 12 |
| WOSCitedRecordID | wos001310497000001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAON databaseName: DOAJ Directory of Open Access Journals customDbUrl: eissn: 2169-3536 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000816957 issn: 2169-3536 databaseCode: DOA dateStart: 20130101 isFulltext: true titleUrlDefault: https://www.doaj.org/ providerName: Directory of Open Access Journals – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2169-3536 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000816957 issn: 2169-3536 databaseCode: M~E dateStart: 20130101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07b9swED40QYZ2yKNNETcPcOhYJTLFhzk6jo0OjWEgz42gyCPiIU7gR8f-9h4p5QEEGbIIAiFBJD_x7rsj7w7gZy2ldkpiofI2I1GAwsWeKkgV8WCCQ5erKFz_0eNx7_bWTNpg9RwLg4j58Bkep9u8lx8e_Cq5ymiFK0UKX6_BmtaqCdZ6dqikChJG6jazULc0J_3BgAZBNiAXx5WQoso88kX75CT9bVWVN6I465fR1gd7tg2bLZFk_Qb5HfiEs6_w5VV6wW8w6Qf3mMQZuyAkqImNpmlznE3mq-QOYee5fDQj3squc4w5GzenwhfsZrq8Y8P7GkkwBXaGWaDswtVoeDn4XbQVFApfSbMsog9aGU8mUeg6LZwxPEZpkDARsesllt457nxtXNQaI7EjxbtYqR7HFDRbfYf12cMM94ApVWMpXcSeIQYVXG1SWpia10TPhddlB349zax9bBJl2GxglMY2QNgEhG2B6MBpmv3nR1OW69xA02rbRWNV5aNDHjRZyyKidzIEpSLZ1HUZhBEd2E1QvPpeg0IHDp7AtO2SXFjisTylAqqqH--8tg-fUxcbB8sBrC_nKzyEDf93OV3Mj7K1Ttfzf8Oj_Of9BwSy1MA |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LbxMxEB7RUqlwAFqKCBTwgWO33Xj9iI8hNCoijSIR0t4srz0WOZBEefD7GXu3pRLiwG1l7Wq9_tYz38x4ZgA-1lJqpyQWKocZiQIULvZUQaqIBxMcutxFYTbS43Hv9tZM2mT1nAuDiPnwGZ6nyxzLD0u_S64y2uFKkcLXe_BYCsHLJl3r3qWSekgYqdvaQt3SXPQHA_oMsgK5OK-EFFVmkn_0Ty7T3_ZV-UsYZw0zfP6fc3sBz1oqyfoN9kfwCBfH8PRBgcGXMOkHt0oCjX0jLGiIDecpPM4m611yiLDr3ECaEXNls5xlzsbNufANu5lvf7DLnzWSaArsM2aRcgLfh5fTwVXR9lAofCXNtog-aGU8GUWh67RwxvAYpUFCRcSul1h657jztXFRa4zEjxTvYqV6HFPabPUK9hfLBb4GplSNpXQRe4Y4VHC1SYVhal4TQRdelx04u1tZu2pKZdhsYpTGNkDYBIRtgejAp7T697emOtd5gJbVttvGqspHhzxospdFRO9kCEpFsqrrMggjOnCSoHjwvgaFDpzegWnbTbmxxGR5KgZUVW_-8dgHOLyaXo_s6Mv461t4kqbbuFtOYX-73uE7OPC_tvPN-n3-834DNOLV4Q |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Adaptive+Scaling+Filter+Pruning+Method+for+Vision+Networks+With+Embedded+Devices&rft.jtitle=IEEE+access&rft.au=Ko%2C+Hyunjun&rft.au=Kang%2C+Jin-Ku&rft.au=Kim%2C+Yongwoo&rft.date=2024&rft.issn=2169-3536&rft.eissn=2169-3536&rft.volume=12&rft.spage=123771&rft.epage=123781&rft_id=info:doi/10.1109%2FACCESS.2024.3454329&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_ACCESS_2024_3454329 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2169-3536&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2169-3536&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2169-3536&client=summon |