RobustNPR: Evaluating the robustness of neural program repair models.
Saved in:
| Title: | RobustNPR: Evaluating the robustness of neural program repair models. |
|---|---|
| Authors: | Ge, Hongliang, Zhong, Wenkang, Li, Chuanyi, Ge, Jidong, Hu, Hao, Luo, Bin |
| Source: | Journal of Software: Evolution & Process; Apr2024, Vol. 36 Issue 4, p1-16, 16p |
| Subject Terms: | SUCCESS |
| Abstract: | Due to the high cost of repairing defective programs, many researches focus on automatic program repair (APR). In recent years, the new trend of APR is to apply neural networks to mine the relations between defective programs and corresponding patches automatically, which is known as neural program repair (NPR). The community, however, ignores some important properties that could impact the applicability of NPR systems, such as robustness. For semantic‐identical buggy programs, NPR systems may produce totally different patches. In this paper, we propose an evaluation tool named RobustNPR, the first NPR robustness evaluation tool. RobustNPR employs several mutators to generate semantic‐identical mutants of defective programs. For an original defective program and its mutant, it checks two aspects of NPR: (a) Can NPR fix mutants when it can fix the original defective program? and (b) can NPR generate semantic‐identical patches for the original program and the mutant? Then, we evaluate four SOTA NPR models and analyze the results. From the results, we find that even for the best‐performing model, 20.16% of the repair success is unreliable, which indicates that the robustness of NPR is not perfect. In addition, we find that the robustness of NPR is correlated with model settings and other factors. [ABSTRACT FROM AUTHOR] |
| Copyright of Journal of Software: Evolution & Process is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Complementary Index |
Full Text Finder 
Nájsť tento článok vo Web of Science Be the first to leave a comment!