Adaptive Scaling Filter Pruning Method for Vision Networks With Embedded Devices

Owing to improvements in computing power, deep learning technology using convolutional neural networks (CNNs) has recently been used in various fields. However, using CNNs on edge devices is challenging because of the large computation required to achieve high performance. To solve this problem, pru...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	IEEE access Ročník 12; s. 123771 - 123781
Hlavní autoři:	Ko, Hyunjun, Kang, Jin-Ku, Kim, Yongwoo
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	Piscataway IEEE 2024 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:	Adaptive filters Adaptive systems Artificial neural networks Batch normalization Computer vision convolutional neural network Convolutional neural networks Datasets Deep learning Filtering algorithms Inference inference time Information filters Machine learning network compression Pruning Quantization (signal) Redundancy Training
ISSN:	2169-3536, 2169-3536
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Abstract	Owing to improvements in computing power, deep learning technology using convolutional neural networks (CNNs) has recently been used in various fields. However, using CNNs on edge devices is challenging because of the large computation required to achieve high performance. To solve this problem, pruning, which reduces redundant parameters and computations, has been widely studied. However, a conventional pruning method requires two learning processes, which are time-consuming and resource-intensive, and it is difficult to reflect the redundancy in the pruned network because it only performs pruning once on the unpruned network. Therefore, in this paper, we utilize a single learning process and propose an adaptive scaling method that dynamically adjusts the size of the network to reflect the changing redundancy in the pruned network. To verify the performance of each method, we compare the performance of the proposed methods by conducting experiments on various datasets and networks. In our experiments using the ImageNet dataset on ResNet-50, pruning FLOPs by 50.1% and 74.0% resulted in a decrease in top-1 accuracy by 0.92% and 3.38%, respectively, and improved inference time by 26.4% and 58.9%, respectively. In addition, pruning FLOPs by 27.37%, 36.84% and 46.41% using the COCO dataset on YOLOv7, reduced mAP(0.5-0.95) by 1.2%, 2.2% and 2.9%, respectively, and improved inference time by 12.9%, 16.9% and19.3%.
AbstractList	Owing to improvements in computing power, deep learning technology using convolutional neural networks (CNNs) has recently been used in various fields. However, using CNNs on edge devices is challenging because of the large computation required to achieve high performance. To solve this problem, pruning, which reduces redundant parameters and computations, has been widely studied. However, a conventional pruning method requires two learning processes, which are time-consuming and resource-intensive, and it is difficult to reflect the redundancy in the pruned network because it only performs pruning once on the unpruned network. Therefore, in this paper, we utilize a single learning process and propose an adaptive scaling method that dynamically adjusts the size of the network to reflect the changing redundancy in the pruned network. To verify the performance of each method, we compare the performance of the proposed methods by conducting experiments on various datasets and networks. In our experiments using the ImageNet dataset on ResNet-50, pruning FLOPs by 50.1% and 74.0% resulted in a decrease in top-1 accuracy by 0.92% and 3.38%, respectively, and improved inference time by 26.4% and 58.9%, respectively. In addition, pruning FLOPs by 27.37%, 36.84% and 46.41% using the COCO dataset on YOLOv7, reduced mAP(0.5-0.95) by 1.2%, 2.2% and 2.9%, respectively, and improved inference time by 12.9%, 16.9% and19.3%.
Author	Kim, Yongwoo Ko, Hyunjun Kang, Jin-Ku
Author_xml	– sequence: 1 givenname: Hyunjun orcidid: 0009-0005-8955-744X surname: Ko fullname: Ko, Hyunjun organization: Department of Electrical and Computer Engineering, Inha University, Incheon, Republic of Korea – sequence: 2 givenname: Jin-Ku orcidid: 0000-0002-3752-3740 surname: Kang fullname: Kang, Jin-Ku organization: Department of Electrical and Computer Engineering, Inha University, Incheon, Republic of Korea – sequence: 3 givenname: Yongwoo orcidid: 0000-0002-1011-2319 surname: Kim fullname: Kim, Yongwoo email: yongwoo.kim@knue.ac.kr organization: Department of Technology Education, Korea National University of Education, Cheongju, Republic of Korea
BookMark	eNpNUctKQzEQDaKgVr9AFwHXrXnfZllqfYAvqI9lSJOJptabmtwq_r23XpHOZmYOc84ZOPtou041IHREyYBSok9H4_FkOh0wwsSACyk401toj1Gl-1xytb0x76LDUuakrWELyWoP3Y-8XTbxE_DU2UWsX_B5XDSQ8X1e1ev1BprX5HFIGT_FElONb6H5Svmt4OfYvOLJ-wy8B4_P4DM6KAdoJ9hFgcO_3kOP55OH8WX_-u7iajy67jsuddMPzldKO6qEp7YSVmsWgtQgJBGBOgnEWcusm2kbqgoCU0IxClwNGTAiOO-hq07XJzs3yxzfbf42yUbzC6T8YmxuoluAUdwFC8xXknERwFnpvVJhyKsZ8UKLVuuk01rm9LGC0ph5WuW6fd9wSphWWvG1I--uXE6lZAj_rpSYdRKmS8KskzB_SbSs444VAWCDoZSQquI_3kGF7g
CODEN	IAECCG
Cites_doi	10.1109/jstsp.2024.3387299 10.1109/ICCV.2017.298 10.1007/978-3-030-58536-5_38 10.1007/s11263-014-0733-5 10.1145/3065386 10.1109/5.726791 10.1109/CVPR.2015.7298594 10.1109/TPAMI.2016.2577031 10.1109/CVPR.2018.00958 10.1007/978-3-319-24574-4_28 10.1109/CVPR42600.2020.00160 10.1007/978-3-030-01270-0_19 10.1109/ICCV48922.2021.00447 10.1016/j.patcog.2020.107461 10.1109/ACCESS.2022.3188323 10.1109/CVPR.2009.5206848 10.48550/arXiv.1802.02611 10.1109/CVPR52729.2023.01544 10.1109/CVPR.2016.90 10.5555/3045118.3045167 10.1109/CVPR.2015.7298965 10.l007/978-3-319-46448-0_2 10.1109/CVPR52729.2023.00721 10.1109/CVPR.2019.01152 10.1109/CVPR52688.2022.01197 10.1109/CVPR.2019.00290 10.1109/VTC2023-Spring57618.2023.10200157 10.1007/978-3-319-10602-1_48 10.24963/ijcai.2018/309 10.1109/CVPR52688.2022.00029
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
DBID	97E ESBDL RIA RIE AAYXX CITATION 7SC 7SP 7SR 8BQ 8FD JG9 JQ2 L7M L~C L~D DOA
DOI	10.1109/ACCESS.2024.3454329
DatabaseName	IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE Xplore Open Access Journals IEEE All-Society Periodicals Package (ASPP) 1998-Present IEEE/IET Electronic Library CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Engineered Materials Abstracts METADEX Technology Research Database Materials Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional DOAJ Directory of Open Access Journals
DatabaseTitle	CrossRef Materials Research Database Engineered Materials Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace METADEX Computer and Information Systems Abstracts Professional
DatabaseTitleList	Materials Research Database
Database_xml	– sequence: 1 dbid: DOA name: DOAJ Directory of Open Access Journals url: https://www.doaj.org/ sourceTypes: Open Website – sequence: 2 dbid: RIE name: IEEE/IET Electronic Library url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	2169-3536
EndPage	123781
ExternalDocumentID	oai_doaj_org_article_63cfae2d75234feca5dd66f837b0d494 10_1109_ACCESS_2024_3454329 10664567
Genre	orig-research
GrantInformation_xml	– fundername: MSIT through the Information Technology Research Center (ITRC) Support Program supervised by the Institute for Information and Communications Technology Planning and Evaluation (IITP) grantid: IITP-2021-0-02052 – fundername: Korean Government through the Ministry of Science and ICT (MSIT), South Korea grantid: 2022R1G1A1007415 – fundername: National Research Foundation of Korea (NRF) Grant
GroupedDBID	0R~ 4.4 5VS 6IK 97E AAJGR ABAZT ABVLG ACGFS ADBBV AGSQL ALMA_UNASSIGNED_HOLDINGS BCNDV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD ESBDL GROUPED_DOAJ IPLJI JAVBF KQ8 M43 M~E O9- OCL OK1 RIA RIE RNS AAYXX CITATION 7SC 7SP 7SR 8BQ 8FD JG9 JQ2 L7M L~C L~D
ID	FETCH-LOGICAL-c359t-fcd769c164d1a74a992ff59e4504f1c5e0caa2acb9af77ef264621e3682e20433
IEDL.DBID	RIE
ISICitedReferencesCount	0
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001310497000001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	2169-3536
IngestDate	Fri Oct 03 12:42:22 EDT 2025 Mon Jun 30 16:34:05 EDT 2025 Sat Nov 29 04:27:04 EST 2025 Wed Aug 27 02:01:30 EDT 2025
IsDoiOpenAccess	true
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Language	English
License	https://creativecommons.org/licenses/by-nc-nd/4.0
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c359t-fcd769c164d1a74a992ff59e4504f1c5e0caa2acb9af77ef264621e3682e20433
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0002-3752-3740 0000-0002-1011-2319 0009-0005-8955-744X
OpenAccessLink	https://ieeexplore.ieee.org/document/10664567
PQID	3102969633
PQPubID	4845423
PageCount	11
ParticipantIDs	doaj_primary_oai_doaj_org_article_63cfae2d75234feca5dd66f837b0d494 crossref_primary_10_1109_ACCESS_2024_3454329 ieee_primary_10664567 proquest_journals_3102969633
PublicationCentury	2000
PublicationDate	20240000 2024-00-00 20240101 2024-01-01
PublicationDateYYYYMMDD	2024-01-01
PublicationDate_xml	– year: 2024 text: 20240000
PublicationDecade	2020
PublicationPlace	Piscataway
PublicationPlace_xml	– name: Piscataway
PublicationTitle	IEEE access
PublicationTitleAbbrev	Access
PublicationYear	2024
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 Kang (ref30) ref12 ref15 ref37 ref36 ref31 ref11 ref33 ref10 ref32 Krizhevsky (ref35) 2009 ref2 ref1 ref39 ref16 ref38 ref19 ref18 Wang (ref34) 2022 Simonyan (ref4) 2014 Li (ref27) 2016 Nonnenmacher (ref29) 2021 Xu (ref14) 2018 ref24 ref23 ref26 ref25 ref20 ref22 ref21 (ref40) 2022 Chen (ref17); 34 ref8 ref7 ref9 ref3 ref6 ref5 Sui (ref28); 34
References_xml	– year: 2018 ident: ref14 article-title: Hybrid pruning: Thinner sparse networks for fast inference on edge devices publication-title: arXiv:1811.00482 – ident: ref1 doi: 10.1109/jstsp.2024.3387299 – ident: ref19 doi: 10.1109/ICCV.2017.298 – ident: ref18 doi: 10.1007/978-3-030-58536-5_38 – year: 2022 ident: ref34 article-title: Trainability preserving neural pruning publication-title: arXiv:2207.12534 – ident: ref38 doi: 10.1007/s11263-014-0733-5 – ident: ref6 doi: 10.1145/3065386 – ident: ref7 doi: 10.1109/5.726791 – ident: ref5 doi: 10.1109/CVPR.2015.7298594 – volume: 34 start-page: 19637 volume-title: Proc. Adv. Neural Inf. Process. Syst. ident: ref17 article-title: Only train once: A one-shot neural network training and pruning framework – year: 2021 ident: ref29 article-title: SOSP: Efficiently capturing global correlations by second-order structured pruning publication-title: arXiv:2110.11395 – start-page: 5122 volume-title: Proc. Int. Conf. Mach. Learn. ident: ref30 article-title: Operation-aware soft channel pruning using differentiable masks – ident: ref10 doi: 10.1109/TPAMI.2016.2577031 – ident: ref20 doi: 10.1109/CVPR.2018.00958 – ident: ref11 doi: 10.1007/978-3-319-24574-4_28 – ident: ref25 doi: 10.1109/CVPR42600.2020.00160 – ident: ref22 doi: 10.1007/978-3-030-01270-0_19 – ident: ref33 doi: 10.1109/ICCV48922.2021.00447 – ident: ref23 doi: 10.1016/j.patcog.2020.107461 – ident: ref15 doi: 10.1109/ACCESS.2022.3188323 – ident: ref36 doi: 10.1109/CVPR.2009.5206848 – ident: ref12 doi: 10.48550/arXiv.1802.02611 – ident: ref31 doi: 10.1109/CVPR52729.2023.01544 – ident: ref3 doi: 10.1109/CVPR.2016.90 – ident: ref39 doi: 10.5555/3045118.3045167 – ident: ref13 doi: 10.1109/CVPR.2015.7298965 – volume: 34 start-page: 24604 volume-title: Proc. Adv. Neural Inf. Process. Syst. (NIPS) ident: ref28 article-title: Chip: Channel independence-based pruning for compact neural networks – ident: ref9 doi: 10.l007/978-3-319-46448-0_2 – ident: ref8 doi: 10.1109/CVPR52729.2023.00721 – volume-title: Filter-Gap year: 2022 ident: ref40 – ident: ref24 doi: 10.1109/CVPR.2019.01152 – ident: ref16 doi: 10.1109/CVPR52688.2022.01197 – ident: ref21 doi: 10.1109/CVPR.2019.00290 – year: 2016 ident: ref27 article-title: Pruning filters for efficient convnets publication-title: arXiv:1608.08710 – ident: ref2 doi: 10.1109/VTC2023-Spring57618.2023.10200157 – volume-title: Learning multiple layers of features from tiny images year: 2009 ident: ref35 – year: 2014 ident: ref4 article-title: Very deep convolutional networks for large-scale image recognition publication-title: arXiv:1409.1556 – ident: ref37 doi: 10.1007/978-3-319-10602-1_48 – ident: ref26 doi: 10.24963/ijcai.2018/309 – ident: ref32 doi: 10.1109/CVPR52688.2022.00029
SSID	ssj0000816957
Score	2.2997766
Snippet	Owing to improvements in computing power, deep learning technology using convolutional neural networks (CNNs) has recently been used in various fields....
SourceID	doaj proquest crossref ieee
SourceType	Open Website Aggregation Database Index Database Publisher
StartPage	123771
SubjectTerms	Adaptive filters Adaptive systems Artificial neural networks Batch normalization Computer vision convolutional neural network Convolutional neural networks Datasets Deep learning Filtering algorithms Inference inference time Information filters Machine learning network compression Pruning Quantization (signal) Redundancy Training
SummonAdditionalLinks	– databaseName: DOAJ Directory of Open Access Journals dbid: DOA link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV27TsMwFLVQxQAD4lFEoCAPjIQmfsZjKa1YqCrx6mY5fogOlKpN-X5sJ0AkBhZWK5Ljc53rcxz7XAAutfLTRrMytVaglDCiU6W1SUu_dilDmeOoLjbBJ5NiNhPTVqmvcCastgeugeszrJ2yyHCvmIizWlFjGHNeV5WZISI6gWZctMRUzMFFzgTljc1Qnon-YDj0I_KCEJFrTCjBkVT-LEXRsb8psfIrL8fFZrwP9hqWCAf12x2ALbs4BLst78AjMB0YtQy5Cj54mH0THM_Dn284XW3CXge8j7WhoSel8DleIIeT-sj3Gr7Mq1c4eiutzzoG3tqYLbrgaTx6HN6lTXmEVGMqqtRpw5nQXu-YXHGihEDOUWEJzYjLNbWZVgopXQrlOLfOUx-GcotZgWy4EYuPQWfxvrAnAOaaUcycyR3HxJqy9IhmyBUquBM5qhJw9YWUXNYuGDKqh0zIGlgZgJUNsAm4CWh-PxosrGODD6xsAiv_CmwCuiEWrf4Y82yPJ6D3FRzZfG9r6UkqCj4_GJ_-R99nYCeMp95q6YFOtdrYc7CtP6r5enURp9onkDzW3g priority: 102 providerName: Directory of Open Access Journals
Title	Adaptive Scaling Filter Pruning Method for Vision Networks With Embedded Devices
URI	https://ieeexplore.ieee.org/document/10664567 https://www.proquest.com/docview/3102969633 https://doaj.org/article/63cfae2d75234feca5dd66f837b0d494
Volume	12
WOSCitedRecordID	wos001310497000001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVAON databaseName: DOAJ Directory of Open Access Journals customDbUrl: eissn: 2169-3536 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000816957 issn: 2169-3536 databaseCode: DOA dateStart: 20130101 isFulltext: true titleUrlDefault: https://www.doaj.org/ providerName: Directory of Open Access Journals – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2169-3536 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000816957 issn: 2169-3536 databaseCode: M~E dateStart: 20130101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV05b9swFH5ogg7t0DNFnaYBh4xVKvE0R8ex0SWGgfTIRlDkI-ohTuAjY357HinlAIoOWQSBkCCKH_ku8n0P4Ch4mjZBtxWi5ZXUMlQ-hFi1pLt8VDoZ3hWbMLPZ8OLCzvtk9ZILg4jl8Bke59uylx-vwjaHymiFa00K3-zAjjG6S9Z6CKjkChJWmZ5ZqKnt99F4TD9BPiCXx0IqKYod-ah9Ckl_X1XlH1Fc9Mv07TN79g7e9IYkG3XIv4cXuPwAr5_QC36E-Sj66yzO2DkhQU1susib42y-2uZwCDsr5aMZ2a3sd8kxZ7PuVPia_Vls_rLJZYskmCI7xSJQ9uDXdPJz_KPqKyhUQSi7qVKIRttALlFsvJHeWp6SsihVLVMTFNbBe-5Da30yBhNZR5o3KPSQY06aFZ9gd3m1xM_AmqCV0Ck2yQiJsW1rY2uehj4TGCXlB_DtfmTddUeU4YqDUVvXAeEyEK4HYgAnefQfHs0s16WBhtX1i8ZpEZJHHg15yzJh8CpGrRP51G0dpZUD2MtQPPleh8IADu7BdP2SXDuyY3mmAhJi_z-vfYFXuYtdgOUAdjerLX6Fl-Fms1ivDou3Ttez28lhmXl3s0TVzA
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1JT9wwFH4qFAl6KGvFtFB84Egg8To-TqeMqIDRSKw3y7GfxRw6oFn6-2s7YZEqDr1FVqw4_uy32e97AIfOxmXjZF0galpwyV1hnfNFHXWX9UIGRZtiE2o47N7f61GbrJ5zYRAxXz7D4_SYz_L9o1ukUFnc4VJGha-W4KPgnJZNutZLSCXVkNBCtdxCValPev1-_I3oBVJ-zLjgLFuSr_on0_S3dVX-EcZZwwzW_3NsG_C5NSVJr8F-Ez7gZAs-vSEY3IZRz9unJNDIVcQiNpHBOB2Pk9F0kQIi5DIXkCbRciW3OcucDJt74TNyN54_kNPfNUbR5MlPzCJlB24Gp9f9s6KtoVA4JvS8CM4rqV10inxlFbda0xCERi5KHionsHTWUutqbYNSGKJ9JGmFTHYpprRZ9gWWJ48T3AVSOSmYDL4KinH0dV0qXdLQtYnCKAjbgaPnmTVPDVWGyS5GqU0DhElAmBaIDvxIs__yauK5zg1xWk27bYxkLlikXkV_mQd0VngvZYhedV16rnkHdhIUb77XoNCBvWcwTbspZyZasjSRATH29Z1uB7B6dn15YS5-Dc-_wVoabhNu2YPl-XSB-7Di_szHs-n3vPL-AuU41u0
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Adaptive+Scaling+Filter+Pruning+Method+for+Vision+Networks+With+Embedded+Devices&rft.jtitle=IEEE+access&rft.au=Ko%2C+Hyunjun&rft.au=Kang%2C+Jin-Ku&rft.au=Kim%2C+Yongwoo&rft.date=2024&rft.pub=IEEE&rft.eissn=2169-3536&rft.volume=12&rft.spage=123771&rft.epage=123781&rft_id=info:doi/10.1109%2FACCESS.2024.3454329&rft.externalDocID=10664567
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2169-3536&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2169-3536&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2169-3536&client=summon