Evaluating the Accuracy of Fault Localization Techniques
We investigate claims and assumptions made in several recent papers about fault localization (FL) techniques. Most of these claims have to do with evaluating FL accuracy. Our investigation centers on a new subject program having properties useful for FL experiments. We find that Tarantula (Jones et...
Uloženo v:
| Vydáno v: | 2009 IEEE/ACM International Conference on Automated Software Engineering s. 76 - 87 |
|---|---|
| Hlavní autoři: | , , , |
| Médium: | Konferenční příspěvek |
| Jazyk: | angličtina |
| Vydáno: |
IEEE
01.11.2009
|
| Témata: | |
| ISBN: | 1424452597, 9781424452590 |
| ISSN: | 1938-4300 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | We investigate claims and assumptions made in several recent papers about fault localization (FL) techniques. Most of these claims have to do with evaluating FL accuracy. Our investigation centers on a new subject program having properties useful for FL experiments. We find that Tarantula (Jones et al.) works well on the program, and we show weak support for the assertion that coverage-based test suites help Tarantula to localize faults. Baudry et al. used automatically-generated mutants to evaluate the accuracy of an FL technique that generates many distinct scores for program locations. We find no evidence to suggest that the use of mutants for this purpose is invalid. However, we find evidence that the standard method for evaluating FL accuracy is unfairly biased toward techniques that generate many distinct scores, and we propose a fairer method of accuracy evaluation. Finally, Denmat et al. suggest that data mining techniques may apply to FL. We investigate this suggestion with the data mining tool Weka, using standard techniques for evaluating the accuracy of data mining classifiers. We find that standard classifiers suffer from the class imbalance problem. However, we find that adding cost information improves accuracy. |
|---|---|
| AbstractList | We investigate claims and assumptions made in several recent papers about fault localization (FL) techniques. Most of these claims have to do with evaluating FL accuracy. Our investigation centers on a new subject program having properties useful for FL experiments. We find that Tarantula (Jones et al.) works well on the program, and we show weak support for the assertion that coverage-based test suites help Tarantula to localize faults. Baudry et al. used automatically-generated mutants to evaluate the accuracy of an FL technique that generates many distinct scores for program locations. We find no evidence to suggest that the use of mutants for this purpose is invalid. However, we find evidence that the standard method for evaluating FL accuracy is unfairly biased toward techniques that generate many distinct scores, and we propose a fairer method of accuracy evaluation. Finally, Denmat et al. suggest that data mining techniques may apply to FL. We investigate this suggestion with the data mining tool Weka, using standard techniques for evaluating the accuracy of data mining classifiers. We find that standard classifiers suffer from the class imbalance problem. However, we find that adding cost information improves accuracy. |
| Author | Dhandapani, T. Ali, S. Andrews, J.H. Wantao Wang |
| Author_xml | – sequence: 1 givenname: S. surname: Ali fullname: Ali, S. organization: Dept. of Comput. Sci., Univ. of Western Ontario, London, ON, Canada – sequence: 2 givenname: J.H. surname: Andrews fullname: Andrews, J.H. organization: Dept. of Comput. Sci., Univ. of Western Ontario, London, ON, Canada – sequence: 3 givenname: T. surname: Dhandapani fullname: Dhandapani, T. organization: Dept. of Comput. Sci., Univ. of Western Ontario, London, ON, Canada – sequence: 4 surname: Wantao Wang fullname: Wantao Wang organization: Dept. of Comput. Sci., Univ. of Western Ontario, London, ON, Canada |
| BookMark | eNotj0tLw0AUhUesYFuzculm_kDinVdm7jKUVIWAC-u6TOZhIzHRPIT66w3o2ZzFd_jgbMiq67tAyC2DjDHA--KlzDgAZgYvSILagM5RCYNMXpINk1xKxRXqFVkzFCaVAuCaJOP4DksWhoatiSm_bTvbqene6HQKtHBuHqw70z7SvZ3biVa9s23zs0z6jh6CO3XN1xzGG3IVbTuG5L-35HVfHnaPafX88LQrqtQKUFPq61xC7r1ElUPNXfQaQEQuuJYKjMZgLRMBYp1rrx1THnnNc8sMc1GhEVty9-dtQgjHz6H5sMP5qKRgy2PxC2wMSTM |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.1109/ASE.2009.89 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan All Online (POP All Online) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISBN | 9780769538914 0769538916 |
| EndPage | 87 |
| ExternalDocumentID | 5431780 |
| Genre | orig-research |
| GroupedDBID | 29I 6IE 6IF 6IH 6IK 6IL 6IM 6IN 6J9 AAJGR AAWTH ABLEC ACREN ADYOE ADZIZ AFYQB ALMA_UNASSIGNED_HOLDINGS AMTXH APO BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IPLJI M43 OCL RIE RIL |
| ID | FETCH-LOGICAL-a305t-db6406dd49560b2cfd7003f2327450879eaa13e0fb67d7c15d92b26a181cf5983 |
| IEDL.DBID | RIE |
| ISBN | 1424452597 9781424452590 |
| ISICitedReferencesCount | 34 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000278137400007&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1938-4300 |
| IngestDate | Wed Aug 27 08:36:35 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a305t-db6406dd49560b2cfd7003f2327450879eaa13e0fb67d7c15d92b26a181cf5983 |
| PageCount | 12 |
| ParticipantIDs | ieee_primary_5431780 |
| PublicationCentury | 2000 |
| PublicationDate | 2009-11 |
| PublicationDateYYYYMMDD | 2009-11-01 |
| PublicationDate_xml | – month: 11 year: 2009 text: 2009-11 |
| PublicationDecade | 2000 |
| PublicationTitle | 2009 IEEE/ACM International Conference on Automated Software Engineering |
| PublicationTitleAbbrev | ASE |
| PublicationYear | 2009 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0000452981 ssj0051577 ssj0001130041 |
| Score | 1.9648106 |
| Snippet | We investigate claims and assumptions made in several recent papers about fault localization (FL) techniques. Most of these claims have to do with evaluating... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 76 |
| SubjectTerms | Computer science Costs Data analysis Data mining Fault localization Genetic mutations Humans mutation analysis Programming profession Software engineering Software testing |
| Title | Evaluating the Accuracy of Fault Localization Techniques |
| URI | https://ieeexplore.ieee.org/document/5431780 |
| WOSCitedRecordID | wos000278137400007&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxGAy1ePBUtRXf5ODR2Owrj2ORFg9SClboreS1IshW2l2h_94vabZ68OJtd08h4ctMvuzMIHRXMG-ZojOSAzqTPGOGKMlKUjDhVJo5S4UJYRN8OhWLhZx10P1eC-OcCz-fuQf_GO7y7co0vlU29LptLuCAfsA532m19v0Ubw0uo4Yy9FeS4CXV7soA2yGFEfiKgCFR2oq8CuD_vPV-iu80CvkSKoejl_HO1dIHwf8KYAn4M-n9b-THaPAj5MOzPUSdoI6rTlGvTXLAsbD7SIyj6Xf1hoER4pExzVqZLV6VeKKajxo_e8yLmk08b41fNwP0OhnPH59IzFQgCiq7JlYzgHBrw7lIp6a0HOq6BF7Fc-BqXDqlkszRUjNuuUkKK1OdMgVEwJSFFNkZ6laryp0jDFSHSSe5d42D5aY6mHkBBdAitzZ1F6jvJ2L5ubPNWMY5uPz78xU6Chc1QeZ3jbr1unE36NB81e-b9W1Y62_Zop7c |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NSwMxFHyUKuipait-m4NH1-5nPo5FWirWUrBCbyWbZEWQVtpdwX_vS5qtHrx4291TSEhm3svODMBNRq1lSp4EKaJzkCZUBVLQIsgoNzJOjA65cmETbDzms5mYNOB2q4Uxxrifz8ydfXR3-XqpKtsq61rdNuNYoO9kaRpHG7XWtqNizcGFV1G6Dkvk3KTqcxmB2-UwImPhOKgwrGVeGVYArHZ_8u-hl_JFoej2nvsbX0sbBf8rgsUh0KD1v7EfQOdHykcmW5A6hIZZHEGrznIgfmu3gfe97ffilSAnJD2lqpVUX2RZkIGs3ksysqjnVZtkWlu_rjvwMuhP74eBT1UIJO7tMtA5RRDX2lVGeawKzXBnF8isWIpsjQkjZZSYsMgp00xFmRZxHlOJVEAVmeDJMTQXy4U5AYJkhwojmPWNwwUPc2fnhSQg56nWsTmFtp2I-cfGOGPu5-Ds78_XsDecPo3mo4fx4znsu2sbJ_q7gGa5qswl7KrP8m29unLr_g1YYaIj |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2009+IEEE%2FACM+International+Conference+on+Automated+Software+Engineering&rft.atitle=Evaluating+the+Accuracy+of+Fault+Localization+Techniques&rft.au=Ali%2C+S.&rft.au=Andrews%2C+J.H.&rft.au=Dhandapani%2C+T.&rft.au=Wantao+Wang&rft.date=2009-11-01&rft.pub=IEEE&rft.isbn=9781424452590&rft.issn=1938-4300&rft.spage=76&rft.epage=87&rft_id=info:doi/10.1109%2FASE.2009.89&rft.externalDocID=5431780 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1938-4300&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1938-4300&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1938-4300&client=summon |