Mutation-based Fault Localization of Deep Neural Networks
Deep neural networks (DNNs) are susceptible to bugs, just like other types of software systems. A significant uptick in using DNN, and its applications in wide-ranging areas, including safety-critical systems, warrant extensive research on software engineering tools for improving the reliability of...
Uloženo v:
| Vydáno v: | IEEE/ACM International Conference on Automated Software Engineering : [proceedings] s. 1301 - 1313 |
|---|---|
| Hlavní autoři: | , , , |
| Médium: | Konferenční příspěvek |
| Jazyk: | angličtina |
| Vydáno: |
IEEE
11.09.2023
|
| Témata: | |
| ISSN: | 2643-1572 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Deep neural networks (DNNs) are susceptible to bugs, just like other types of software systems. A significant uptick in using DNN, and its applications in wide-ranging areas, including safety-critical systems, warrant extensive research on software engineering tools for improving the reliability of DNN-based systems. One such tool that has gained significant attention in the recent years is DNN fault localization. This paper revisits mutation-based fault localization in the context of DNN models and proposes a novel technique, named deepmufl, applicable to a wide range of DNN models. We have implemented deepmufl and have evaluated its effectiveness using 109 bugs obtained from StackOverflow. Our results show that deepmufl detects 53/109 of the bugs by ranking the buggy layer in top-1 position, outperforming state-of-the-art static and dynamic DNN fault localization systems that are also designed to target the class of bugs supported by deepmufl. Moreover, we observed that we can halve the fault localization time for a pre-trained model using mutation selection, yet losing only 7.55% of the bugs localized in ton-1 position. |
|---|---|
| AbstractList | Deep neural networks (DNNs) are susceptible to bugs, just like other types of software systems. A significant uptick in using DNN, and its applications in wide-ranging areas, including safety-critical systems, warrant extensive research on software engineering tools for improving the reliability of DNN-based systems. One such tool that has gained significant attention in the recent years is DNN fault localization. This paper revisits mutation-based fault localization in the context of DNN models and proposes a novel technique, named deepmufl, applicable to a wide range of DNN models. We have implemented deepmufl and have evaluated its effectiveness using 109 bugs obtained from StackOverflow. Our results show that deepmufl detects 53/109 of the bugs by ranking the buggy layer in top-1 position, outperforming state-of-the-art static and dynamic DNN fault localization systems that are also designed to target the class of bugs supported by deepmufl. Moreover, we observed that we can halve the fault localization time for a pre-trained model using mutation selection, yet losing only 7.55% of the bugs localized in ton-1 position. |
| Author | Rajan, Hridesh Ghanbari, Ali Arshad, Muhammad Arbab Thomas, Deepak-George |
| Author_xml | – sequence: 1 givenname: Ali surname: Ghanbari fullname: Ghanbari, Ali email: alig@iastate.edu organization: Iowa State University,Dept. of Computer Science,Ames,Iowa,USA – sequence: 2 givenname: Deepak-George surname: Thomas fullname: Thomas, Deepak-George email: dgthomas@iastate.edu organization: Iowa State University,Dept. of Computer Science,Ames,Iowa,USA – sequence: 3 givenname: Muhammad Arbab surname: Arshad fullname: Arshad, Muhammad Arbab email: arbab@iastate.edu organization: Iowa State University,Dept. of Computer Science,Ames,Iowa,USA – sequence: 4 givenname: Hridesh surname: Rajan fullname: Rajan, Hridesh email: hridesh@iastate.edu organization: Iowa State University,Dept. of Computer Science,Ames,Iowa,USA |
| BookMark | eNotj81Kw0AUhUdRsK19Al3kBZLeufO_LLVVIdWFui6TzB2IxqTkB9Gnb1BXH-d8cODM2UXTNsTYDYeMc3Cr9ctWaUSXIaDIALjhZ2zpjLNCgUDntDxnM9RSpFwZvGLzvn8HUFMwM-b24-CHqm3SwvcUkp0f6yHJ29LX1c-vSNqY3BEdkycaO19PGL7a7qO_ZpfR1z0t_7lgb7vt6-YhzZ_vHzfrPPVo5ZBqYUuwnLgFHqILlhuU2pvCOwOEGG0MMRQqRqkLY8gapb0tjZI2TAWKBbv9262I6HDsqk_ffR844HTQSXECwg5I2A |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.1109/ASE56229.2023.00171 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Xplore POP ALL IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISBN | 9798350329964 |
| EISSN | 2643-1572 |
| EndPage | 1313 |
| ExternalDocumentID | 10298394 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: National Science Foundation grantid: 2127309,2223812,2120448,1934884 funderid: 10.13039/100000001 |
| GroupedDBID | 6IE 6IF 6IH 6IK 6IL 6IM 6IN 6J9 AAJGR AAWTH ABLEC ACREN ADYOE ADZIZ AFYQB ALMA_UNASSIGNED_HOLDINGS AMTXH BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IPLJI M43 OCL RIE RIL |
| ID | FETCH-LOGICAL-a284t-638c081e1801df9d817246a7ba970e22f8fdfdb5ff46b77e8756a8c7548d6b723 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 15 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001103357200104&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:32:28 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a284t-638c081e1801df9d817246a7ba970e22f8fdfdb5ff46b77e8756a8c7548d6b723 |
| PageCount | 13 |
| ParticipantIDs | ieee_primary_10298394 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-Sept.-11 |
| PublicationDateYYYYMMDD | 2023-09-11 |
| PublicationDate_xml | – month: 09 year: 2023 text: 2023-Sept.-11 day: 11 |
| PublicationDecade | 2020 |
| PublicationTitle | IEEE/ACM International Conference on Automated Software Engineering : [proceedings] |
| PublicationTitleAbbrev | ASE |
| PublicationYear | 2023 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0051577 ssib057256115 |
| Score | 2.3629565 |
| Snippet | Deep neural networks (DNNs) are susceptible to bugs, just like other types of software systems. A significant uptick in using DNN, and its applications in... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 1301 |
| SubjectTerms | Artificial neural networks Benchmark testing Computer bugs Data models Deep Neural Network Fault Localization Location awareness Mutation Reliability engineering Software systems |
| Title | Mutation-based Fault Localization of Deep Neural Networks |
| URI | https://ieeexplore.ieee.org/document/10298394 |
| WOSCitedRecordID | wos001103357200104&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlZ27T8MwEMYtihiYyqOItzywGmLHzsUjglYMpaoESN0qO7YlpKqt2oS_n7ObthMDUyJP1iWffufHd0fIQ5lnVSFVzpxSgkkLjmmDwitsVYq47a9D6loyhNGonEz0uDWrJy-M9z5dPvOP8TWd5btF1cStMlS40Ah02SEdANiYtbY_jwKEN-e73Bc5DdCWGeKZfnr-6CPqRfSmiFjUlCfT_L6hSuLJoPvPmZyQ3t6ZR8c75pySAz8_I91tawbaKvWc6Pdmc8bOIqYcHZhmVtNhBFdrvKSLQF-9X9JYnsPM8JHug6975GvQ_3x5Y22XBGYQLTVDAVXIdc-RNS5oV2JKIgsD1mjIvBChDC44q0KQhQXwuEApTFkBLlUcDoj8ghzOF3N_SaiVeUxQigyEk5mUGvVpgzSoWQFWqSvSi6GYLjeFMKbbKFz_MX5DjmO04_UKzm_JYb1q_B05qn7q7_XqPn2-X2rml6k |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlZ3PS8MwFMeDTkFP88fE3-bgtdqkSdMcRTcmdmPghN1G0iQgjHVsrX-_L1m3nTx4askpvPbL5-XH9z2EHrMkLlLGk8hwTiOmhYmkAuGlusio3_aXLnQtycVwmE0mctSY1YMXxlobLp_ZJ_8azvJNWdR-qwwUTiUAne2jA84YJWu71ub34QLwTcg2-wVSC9EUGiKxfH757ALsqXenUF_WlATb_K6lSiBKr_3PuZygzs6bh0db6pyiPTs_Q-1NcwbcaPUcyUG9PmWPPKgM7ql6VuHco6uxXuLS4TdrF9gX6FAzeIQb4asO-up1x6_9qOmTECmASxWBhAoguyVAG-OkySApYakSWkkRW0pd5owzmjvHUi2EhSVKqrJCwGLFwABNLlBrXs7tJcKaJT5FSWNBDYsZk6BQ7ZgC1VKhOb9CHR-K6WJdCmO6icL1H-MP6Kg_HuTT_H34cYOOfeT9ZQtCblGrWtb2Dh0WP9X3ankfPuUvgrOa8A |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=IEEE%2FACM+International+Conference+on+Automated+Software+Engineering+%3A+%5Bproceedings%5D&rft.atitle=Mutation-based+Fault+Localization+of+Deep+Neural+Networks&rft.au=Ghanbari%2C+Ali&rft.au=Thomas%2C+Deepak-George&rft.au=Arshad%2C+Muhammad+Arbab&rft.au=Rajan%2C+Hridesh&rft.date=2023-09-11&rft.pub=IEEE&rft.eissn=2643-1572&rft.spage=1301&rft.epage=1313&rft_id=info:doi/10.1109%2FASE56229.2023.00171&rft.externalDocID=10298394 |