Evaluating the Accuracy of Fault Localization Techniques

We investigate claims and assumptions made in several recent papers about fault localization (FL) techniques. Most of these claims have to do with evaluating FL accuracy. Our investigation centers on a new subject program having properties useful for FL experiments. We find that Tarantula (Jones et...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:2009 IEEE/ACM International Conference on Automated Software Engineering s. 76 - 87
Hlavní autoři: Ali, S., Andrews, J.H., Dhandapani, T., Wantao Wang
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 01.11.2009
Témata:
Computer science
Costs
Data analysis
Data mining
Fault localization
Genetic mutations
Humans
mutation analysis
Programming profession
Software engineering
Software testing
ISBN:1424452597, 9781424452590
ISSN:1938-4300
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:We investigate claims and assumptions made in several recent papers about fault localization (FL) techniques. Most of these claims have to do with evaluating FL accuracy. Our investigation centers on a new subject program having properties useful for FL experiments. We find that Tarantula (Jones et al.) works well on the program, and we show weak support for the assertion that coverage-based test suites help Tarantula to localize faults. Baudry et al. used automatically-generated mutants to evaluate the accuracy of an FL technique that generates many distinct scores for program locations. We find no evidence to suggest that the use of mutants for this purpose is invalid. However, we find evidence that the standard method for evaluating FL accuracy is unfairly biased toward techniques that generate many distinct scores, and we propose a fairer method of accuracy evaluation. Finally, Denmat et al. suggest that data mining techniques may apply to FL. We investigate this suggestion with the data mining tool Weka, using standard techniques for evaluating the accuracy of data mining classifiers. We find that standard classifiers suffer from the class imbalance problem. However, we find that adding cost information improves accuracy.
ISBN:1424452597
9781424452590
ISSN:1938-4300
DOI:10.1109/ASE.2009.89