Boosting Adversarial Transferability With Learnable Patch-Wise Masks
Adversarial examples have attracted widespread attention in security-critical applications because of their transferability across different models. Although many methods have been proposed to boost adversarial transferability, a gap still exists between capabilities and practical demand. In this ar...
Saved in:
| Published in: | IEEE transactions on multimedia Vol. 26; pp. 3778 - 3787 |
|---|---|
| Main Authors: | , |
| Format: | Journal Article |
| Language: | English |
| Published: |
Piscataway
IEEE
2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Subjects: | |
| ISSN: | 1520-9210, 1941-0077 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | Adversarial examples have attracted widespread attention in security-critical applications because of their transferability across different models. Although many methods have been proposed to boost adversarial transferability, a gap still exists between capabilities and practical demand. In this article, we argue that the model-specific discriminative regions are a key factor causing overfitting to the source model, and thus reducing the transferability to the target model. For that, a patch-wise mask is utilized to prune the model-specific regions when calculating adversarial perturbations. To accurately localize these regions, we present a learnable approach to automatically optimize the mask. Specifically, we simulate the target models in our framework, and adjust the patch-wise mask according to the feedback of the simulated models. To improve the efficiency, the differential evolutionary (DE) algorithm is utilized to search for patch-wise masks for a specific image. During iterative attacks, the learned masks are applied to the image to drop out the patches related to model-specific regions, thus making the gradients more generic and improving the adversarial transferability. The proposed approach is a preprocessing method and can be integrated with existing methods to further boost the transferability. Extensive experiments on the ImageNet dataset demonstrate the effectiveness of our method. We incorporate the proposed approach with existing methods to perform ensemble attacks and achieve an average success rate of 93.01% against seven advanced defense methods, which can effectively enhance the state-of-the-art transfer-based attack performance. |
|---|---|
| AbstractList | Adversarial examples have attracted widespread attention in security-critical applications because of their transferability across different models. Although many methods have been proposed to boost adversarial transferability, a gap still exists between capabilities and practical demand. In this article, we argue that the model-specific discriminative regions are a key factor causing overfitting to the source model, and thus reducing the transferability to the target model. For that, a patch-wise mask is utilized to prune the model-specific regions when calculating adversarial perturbations. To accurately localize these regions, we present a learnable approach to automatically optimize the mask. Specifically, we simulate the target models in our framework, and adjust the patch-wise mask according to the feedback of the simulated models. To improve the efficiency, the differential evolutionary (DE) algorithm is utilized to search for patch-wise masks for a specific image. During iterative attacks, the learned masks are applied to the image to drop out the patches related to model-specific regions, thus making the gradients more generic and improving the adversarial transferability. The proposed approach is a preprocessing method and can be integrated with existing methods to further boost the transferability. Extensive experiments on the ImageNet dataset demonstrate the effectiveness of our method. We incorporate the proposed approach with existing methods to perform ensemble attacks and achieve an average success rate of 93.01% against seven advanced defense methods, which can effectively enhance the state-of-the-art transfer-based attack performance. |
| Author | Wei, Xingxing Zhao, Shiji |
| Author_xml | – sequence: 1 givenname: Xingxing orcidid: 0000-0002-0778-8377 surname: Wei fullname: Wei, Xingxing email: xxwei@buaa.edu.cn organization: Institute of Artificial Intelligence, Beihang University, Beijing, China – sequence: 2 givenname: Shiji orcidid: 0000-0001-6033-6049 surname: Zhao fullname: Zhao, Shiji email: zhaoshiji123@buaa.edu.cn organization: Institute of Artificial Intelligence, Beihang University, Beijing, China |
| BookMark | eNp9kD1PwzAURS0EEqWwMzBEYk55tmunHkv5lFrBUNTRenFs6hKSYrtI_fekagfEwPTecM_V1Tkjx03bWEIuKQwoBXUzn80GDBgfcE6FEHBEelQNaQ5QFMfdLxjkilE4JWcxrgDoUEDRI3e3bRuTb96zcfVtQ8Tgsc7mAZvobMDS1z5ts4VPy2xqMTRY1jZ7xWSW-cJHm80wfsRzcuKwjvbicPvk7eF-PnnKpy-Pz5PxNDdMsZRLgW5Ulk5RxwrHsRxVVllnJXPGgVFGDE3BnShV2a0FcJUBR5UxAqWrWMX75Hrfuw7t18bGpFftpttUR80UByUlcNal5D5lQhtjsE4bnzD5tkkBfa0p6J0x3RnTO2P6YKwD4Q-4Dv4Tw_Y_5GqPeGvtrzgTVILkP_QmeVk |
| CODEN | ITMUF8 |
| CitedBy_id | crossref_primary_10_1109_TIP_2025_3586485 crossref_primary_10_1016_j_imavis_2025_105722 crossref_primary_10_1007_s11263_025_02552_x crossref_primary_10_1109_TMM_2024_3521769 |
| Cites_doi | 10.1109/CVPR.2019.00284 10.1088/1742-5468/ac9830 10.1109/CVPR.2019.00444 10.1109/TPAMI.2022.3231886 10.1109/ICCV.2015.169 10.1609/aaai.v33i01.33011028 10.1109/ICCV48922.2021.01172 10.1007/978-3-031-19772-7_34 10.1109/CVPR42600.2020.00034 10.1109/tmm.2022.3173533 10.1109/ICCV48922.2021.01585 10.1109/CVPR.2017.243 10.1201/9781351251389-8 10.1109/CVPR46437.2021.00891 10.1109/TMM.2021.3124083 10.1177/104649647100200201 10.1109/TMM.2021.3079723 10.1007/978-3-031-19772-7_32 10.1109/CVPR.2016.90 10.48550/arXiv.2010.11929 10.1109/CVPR46437.2021.00196 10.1109/CVPR52688.2022.01457 10.1109/CVPR.2018.00957 10.14722/ndss.2018.23198 10.1109/CVPR.2016.308 10.1109/TPAMI.2022.3176760 10.5555/3298023.3298188 10.1109/ICCV48922.2021.00754 10.1007/978-3-540-68830-3 10.1109/CVPR42600.2020.00124 10.1109/CVPR52688.2022.01456 10.1109/CVPR.2019.00095 10.1038/30918 10.1109/ICCV48922.2021.01204 10.1109/ICCV48922.2021.00063 10.1109/CVPR.2018.00191 10.1109/CVPR.2015.7298965 10.1109/tmm.2023.3255742 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024 |
| DBID | 97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D |
| DOI | 10.1109/TMM.2023.3315550 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Technology Research Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE/IET Electronic Library (IEL) (UW System Shared) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Statistics Computer Science |
| EISSN | 1941-0077 |
| EndPage | 3787 |
| ExternalDocumentID | 10_1109_TMM_2023_3315550 10251606 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Fundamental Research Funds for the Central Universities funderid: 10.13039/501100012226 – fundername: National Natural Science Foundation of China; Project of the National Natural Science Foundation of China grantid: 62076018 funderid: 10.13039/501100001809 |
| GroupedDBID | -~X 0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD HZ~ H~9 IFIPE IFJZH IPLJI JAVBF LAI M43 O9- OCL P2P PQQKQ RIA RIE RNS TN5 VH1 ZY4 AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D |
| ID | FETCH-LOGICAL-c292t-65af8bbf91f27f3ab8de9efe62fcf0c9c54c73f5b9b52000fdc0f19cc5a6fd2d3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 9 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001175134300024&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1520-9210 |
| IngestDate | Mon Jun 30 05:42:15 EDT 2025 Sat Nov 29 03:10:12 EST 2025 Tue Nov 18 21:58:05 EST 2025 Wed Aug 27 02:11:30 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c292t-65af8bbf91f27f3ab8de9efe62fcf0c9c54c73f5b9b52000fdc0f19cc5a6fd2d3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0001-6033-6049 0000-0002-0778-8377 |
| PQID | 2930966032 |
| PQPubID | 75737 |
| PageCount | 10 |
| ParticipantIDs | crossref_primary_10_1109_TMM_2023_3315550 ieee_primary_10251606 crossref_citationtrail_10_1109_TMM_2023_3315550 proquest_journals_2930966032 |
| PublicationCentury | 2000 |
| PublicationDate | 20240000 2024-00-00 20240101 |
| PublicationDateYYYYMMDD | 2024-01-01 |
| PublicationDate_xml | – year: 2024 text: 20240000 |
| PublicationDecade | 2020 |
| PublicationPlace | Piscataway |
| PublicationPlace_xml | – name: Piscataway |
| PublicationTitle | IEEE transactions on multimedia |
| PublicationTitleAbbrev | TMM |
| PublicationYear | 2024 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref12 ref14 Tramr (ref25) 2018 ref11 Goodfellow (ref5) 2015 ref17 ref16 ref19 ref18 Xie (ref33) 2018 ref50 ref46 Simonyan (ref20) 2015 ref45 ref48 ref47 ref42 ref41 ref44 ref49 ref8 ref7 Xie (ref32) 2018 ref4 ref3 ref6 ref40 Smilkov (ref21) 2017 Lin (ref13) 2019 ref35 ref34 ref37 ref36 Liu (ref15) 2017 ref31 ref30 ref2 ref39 ref38 Touvron (ref43) 2021 Krizhevsky (ref10) 2012; 25 Chakraborty (ref1) 2008; 143 Szegedy (ref24) 2014 ref23 ref26 ref22 ref28 ref27 ref29 Jia (ref9) 2020 |
| References_xml | – ident: ref34 doi: 10.1109/CVPR.2019.00284 – ident: ref40 doi: 10.1088/1742-5468/ac9830 – ident: ref3 doi: 10.1109/CVPR.2019.00444 – ident: ref50 doi: 10.1109/TPAMI.2022.3231886 – ident: ref4 doi: 10.1109/ICCV.2015.169 – volume: 25 start-page: 1097 volume-title: Proc. Adv. Neural Inf. Process. Syst. year: 2012 ident: ref10 article-title: ImageNet classification with deep convolutional neural networks – start-page: 1 volume-title: Proc. Int. Conf. Learn. Representations year: 2014 ident: ref24 article-title: Intriguing properties of neural networks – ident: ref14 doi: 10.1609/aaai.v33i01.33011028 – start-page: 1 volume-title: Proc. 6th Int. Conf. Learn. Representations year: 2018 ident: ref33 article-title: Mitigating adversarial effects through randomization – start-page: 1 volume-title: Proc. 3rd Int. Conf. Learn. Representations year: 2015 ident: ref5 article-title: Explaining and harnessing adversarial examples – ident: ref42 doi: 10.1109/ICCV48922.2021.01172 – ident: ref44 doi: 10.1007/978-3-031-19772-7_34 – ident: ref19 doi: 10.1109/CVPR42600.2020.00034 – ident: ref47 doi: 10.1109/tmm.2022.3173533 – ident: ref27 doi: 10.1109/ICCV48922.2021.01585 – ident: ref8 doi: 10.1109/CVPR.2017.243 – start-page: 1 volume-title: Proc. Int. Conf. Learn. Representations year: 2020 ident: ref9 article-title: Certified robustness for top-k predictions against adversarial perturbations via randomized smoothing – ident: ref11 doi: 10.1201/9781351251389-8 – ident: ref31 doi: 10.1109/CVPR46437.2021.00891 – ident: ref48 doi: 10.1109/TMM.2021.3124083 – ident: ref7 doi: 10.1177/104649647100200201 – start-page: 1 volume-title: Proc. 3rd IAPR Asian Conf. Pattern Recognit. year: 2015 ident: ref20 article-title: Very deep convolutional networks for large-scale image recognition – start-page: 1 volume-title: Proc. Int. Conf. Learn. Representations year: 2018 ident: ref32 article-title: Mitigating adversarial effects through randomization – start-page: 10347 volume-title: Proc. Int. Conf. Mach. Learn. year: 2021 ident: ref43 article-title: Training data-efficient image transformers & distillation through attention – ident: ref45 doi: 10.1109/TMM.2021.3079723 – ident: ref18 doi: 10.1007/978-3-031-19772-7_32 – year: 2017 ident: ref21 article-title: Smoothgrad: Removing noise by adding noise – ident: ref6 doi: 10.1109/CVPR.2016.90 – ident: ref39 doi: 10.48550/arXiv.2010.11929 – start-page: 1 volume-title: Proc. 6th Int. Conf. Learn. Representations year: 2018 ident: ref25 article-title: Ensemble adversarial training: Attacks and defenses – ident: ref26 doi: 10.1109/CVPR46437.2021.00196 – ident: ref37 doi: 10.1109/CVPR52688.2022.01457 – ident: ref2 doi: 10.1109/CVPR.2018.00957 – ident: ref36 doi: 10.14722/ndss.2018.23198 – ident: ref23 doi: 10.1109/CVPR.2016.308 – ident: ref49 doi: 10.1109/TPAMI.2022.3176760 – ident: ref22 doi: 10.5555/3298023.3298188 – start-page: 1 volume-title: Proc. Int. Conf. Learn. Representations year: 2017 ident: ref15 article-title: Delving into transferable adversarial examples and black-box attacks – ident: ref28 doi: 10.1109/ICCV48922.2021.00754 – volume: 143 volume-title: Advances in Differential Evolution year: 2008 ident: ref1 doi: 10.1007/978-3-540-68830-3 – ident: ref30 doi: 10.1109/CVPR42600.2020.00124 – ident: ref35 doi: 10.1109/CVPR52688.2022.01456 – ident: ref16 doi: 10.1109/CVPR.2019.00095 – ident: ref29 doi: 10.1038/30918 – start-page: 1 volume-title: Proc. Int. Conf. Learn. Representations year: 2019 ident: ref13 article-title: Nesterov accelerated gradient and scale invariance for adversarial attacks – ident: ref41 doi: 10.1109/ICCV48922.2021.01204 – ident: ref38 doi: 10.1109/ICCV48922.2021.00063 – ident: ref12 doi: 10.1109/CVPR.2018.00191 – ident: ref17 doi: 10.1109/CVPR.2015.7298965 – ident: ref46 doi: 10.1109/tmm.2023.3255742 |
| SSID | ssj0014507 |
| Score | 2.4516895 |
| Snippet | Adversarial examples have attracted widespread attention in security-critical applications because of their transferability across different models. Although... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 3778 |
| SubjectTerms | Adaptation models Adversarial Attack Adversarial Transferability DNNs Evolutionary algorithms Iterative methods Masks Perturbation methods Predictive models Statistics Training Visualization |
| Title | Boosting Adversarial Transferability With Learnable Patch-Wise Masks |
| URI | https://ieeexplore.ieee.org/document/10251606 https://www.proquest.com/docview/2930966032 |
| Volume | 26 |
| WOSCitedRecordID | wos001175134300024&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE/IET Electronic Library (IEL) (UW System Shared) customDbUrl: eissn: 1941-0077 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014507 issn: 1520-9210 databaseCode: RIE dateStart: 19990101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV09T8MwED1BxVAGCqWIQkEeWBjc5jvxyFfF0qpDUbtFtmOrFahBTYvEv-fsJKgSAoktgx1Fefa988e9B3AjQi21cjn1MHOjgXAkFUiklGWuH2B-q3iYWbOJeDxO5nM2qYrVbS2MUspePlN982jP8rNcbs1WGc5wZOPICGzvx3FUFmt9HxkEoa2NRj5yKMOFTH0m6bDBdDTqG5vwvu8jfZoS-x0OsqYqPyKxpZdh658fdgxHVR5J7krgT2BPrdrQqj0aSDVl23C4IzjYhqbJLUtp5lN4vM_zwtx6JtaVueBmLBJLXlqtS_3uTzJbbhbEqrCaIisywdC9oLNlociIF69FB16GT9OHZ1qZKlDpMW9Do5DrRAjNXO3F2uciyRRTWkUeguZIJsNAxr4OBRNGkcnRmXS0y6QMeaQzL_PPoLHKV-ociNbGMBRDBqYNgfCk0EmG60PBpSsxbMguDOrfnMpKcdwYX7ylduXhsBSBSQ0waQVMF26_e7yXaht_tO0YIHbalRh0oVdDmVbzsUgxqXGMDqnvXfzS7RKa-Pag3F3pQWOz3qorOJAfiMr62g61LxaT0UY |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3JTsMwEB0hQKIcWEoRZfWBCwe3jrO0PrJVRZCKQxG9RbFjqxWoQU1B4u8ZOymqhEDiloOtRHn2vPEy7wGcy9Aoo72UcszcaCCZohKJlIrM8wPMb3UaZs5sojMYdEcj8VgVq7taGK21u3ymW_bRneVnuXq3W2U4w5GNIyuwvRYGAWdludb3oUEQuupoZCRGBS5lFqeSTLSHcdyyRuEt30cCtUX2SyzkbFV-xGJHML3tf37aDmxVmSS5LKHfhRU9rcP2wqWBVJO2DptLkoN1qNnsshRn3oObqzwv7L1n4nyZi9SORuLoy-hZqeD9SZ4n8zFxOqy2zIo8YvAe0-dJoUmcFi9FA556t8PrPq1sFajigs9pFKamK6URnuEd46eym2mhjY44wsaUUGGgOr4JpZBWk4mZTDHjCaXCNDIZz_x9WJ3mU30AxBhrGYpBAxOHQHIlTTfDFaJMlacwcKgmtBe_OVGV5ri1vnhN3NqDiQSBSSwwSQVMEy6-e7yVeht_tG1YIJbalRg04XgBZVLNyCLBtIZZJVKfH_7S7Qw2-sP4IXm4G9wfQQ3fFJR7LcewOp-96xNYVx-I0OzUDbsvImPUjQ |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Boosting+Adversarial+Transferability+With+Learnable+Patch-Wise+Masks&rft.jtitle=IEEE+transactions+on+multimedia&rft.au=Wei%2C+Xingxing&rft.au=Zhao%2C+Shiji&rft.date=2024&rft.pub=IEEE&rft.issn=1520-9210&rft.volume=26&rft.spage=3778&rft.epage=3787&rft_id=info:doi/10.1109%2FTMM.2023.3315550&rft.externalDocID=10251606 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1520-9210&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1520-9210&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1520-9210&client=summon |