Automated Behavioral Testing of Refactoring Engines

Refactoring is a transformation that preserves the external behavior of a program and improves its internal quality. Usually, compilation errors and behavioral changes are avoided by preconditions determined for each refactoring transformation. However, to formally define these preconditions and tra...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on software engineering Vol. 39; no. 2; pp. 147 - 162
Main Authors: Soares, G., Gheyi, R., Massoni, T.
Format: Journal Article
Language:English
Published: New York IEEE 01.02.2013
IEEE Computer Society
Subjects:
automated testing
Automatic programming
Automation
Code reuse
Computer bugs
Computer programming
Debugging
Engines
Java
Metals
program generation
Refactoring
Software engineering
Software quality
Studies
Testing
Unified modeling language
ISSN:0098-5589, 1939-3520
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Refactoring is a transformation that preserves the external behavior of a program and improves its internal quality. Usually, compilation errors and behavioral changes are avoided by preconditions determined for each refactoring transformation. However, to formally define these preconditions and transfer them to program checks is a rather complex task. In practice, refactoring engine developers commonly implement refactorings in an ad hoc manner since no guidelines are available for evaluating the correctness of refactoring implementations. As a result, even mainstream refactoring engines contain critical bugs. We present a technique to test Java refactoring engines. It automates test input generation by using a Java program generator that exhaustively generates programs for a given scope of Java declarations. The refactoring under test is applied to each generated program. The technique uses SafeRefactor, a tool for detecting behavioral changes, as an oracle to evaluate the correctness of these transformations. Finally, the technique classifies the failing transformations by the kind of behavioral change or compilation error introduced by them. We have evaluated this technique by testing 29 refactorings in Eclipse JDT, NetBeans, and the JastAdd Refactoring Tools. We analyzed 153,444 transformations, and identified 57 bugs related to compilation errors, and 63 bugs related to behavioral changes.
Bibliography:SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
ISSN:0098-5589
1939-3520
DOI:10.1109/TSE.2012.19