A systematic literature review of techniques and metrics to reduce the cost of mutation testing

•Comprehensive classifications for cost reduction goals and cost reduction techniques.•Overview of all selected studies grouped by cost reduction goal.•Characterization of metrics used to measure cost reduction.•Summary, analysis and discussion about cost savings and test effectiveness.•Recommendati...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:The Journal of systems and software Ročník 157; s. 110388
Hlavní autoři: Pizzoleto, Alessandro Viola, Ferrari, Fabiano Cutigi, Offutt, Jeff, Fernandes, Leo, Ribeiro, Márcio
Médium: Journal Article
Jazyk:angličtina
Vydáno: Elsevier Inc 01.11.2019
Témata:
Cost reduction
Mutation analysis
Mutation testing
Systematic review
Systematic review
Mutation analysis
Mutation testing
Cost reduction
ISSN:0164-1212, 1873-1228
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:•Comprehensive classifications for cost reduction goals and cost reduction techniques.•Overview of all selected studies grouped by cost reduction goal.•Characterization of metrics used to measure cost reduction.•Summary, analysis and discussion about cost savings and test effectiveness.•Recommendations for future research. Historically, researchers have proposed and applied many techniques to reduce the cost of mutation testing. It has become difficult to find all techniques and to understand the cost-benefit tradeoffs among them, which is critical to transitioning this technology to practice. This paper extends a prior workshop paper to summarize and analyze the current knowledge about reducing the cost of mutation testing through a systematic literature review. We selected 175 peer-reviewed studies, from which 153 present either original or updated contributions. Our analysis resulted in six main goals for cost reduction and 21 techniques. In the last decade, a growing number of studies explored techniques such as selective mutation, evolutionary algorithms, control-flow analysis, and higher-order mutation. Furthermore, we characterized 18 metrics, with particular interest in the number of mutants to be executed, test cases required, equivalent mutants generated and detected, and mutant execution speedup. We found that cost reduction for mutation is increasingly becoming interdisciplinary, often combining multiple techniques. Additionally, measurements vary even for studies that use the same techniques. Researchers can use our results to find more detailed information about particular techniques, and to design comparable and reproducible experiments.
ISSN:0164-1212
1873-1228
DOI:10.1016/j.jss.2019.07.100