On the Use of Deep Learning Models for Semantic Clone Detection
Saved in:
| Title: | On the Use of Deep Learning Models for Semantic Clone Detection |
|---|---|
| Authors: | Pinku, Subroto Nag, Mondal, Debajyoti, Roy, Chanchal K. |
| Source: | 2024 IEEE International Conference on Software Maintenance and Evolution (ICSME). :512-524 |
| Publication Status: | Preprint |
| Publisher Information: | IEEE, 2024. |
| Publication Year: | 2024 |
| Subject Terms: | Software Engineering (cs.SE), FOS: Computer and information sciences, Computer Science - Software Engineering, Computer Science - Machine Learning, D.2.13, D.2, Machine Learning (cs.LG) |
| Description: | Detecting and tracking code clones can ease various software development and maintenance tasks when changes in a code fragment should be propagated over all its copies. Several deep learning-based clone detection models have appeared in the literature for detecting syntactic and semantic clones, widely evaluated with the BigCloneBench dataset. However, class imbalance and the small number of semantic clones make BigCloneBench less ideal for interpreting model performance. Researchers also use other datasets such as GoogleCodeJam, OJClone, and SemanticCloneBench to understand model generalizability. To overcome the limitations of existing datasets, the GPT-assisted semantic and cross-language clone dataset GPTCloneBench has been released. However, how these models compare across datasets remains unclear. In this paper, we propose a multi-step evaluation approach for five state-of-the-art clone detection models leveraging existing benchmark datasets, including GPTCloneBench, and using mutation operators to study model ability. Specifically, we examine three highly-performing single-language models (ASTNN, GMN, CodeBERT) on BigCloneBench, SemanticCloneBench, and GPTCloneBench, testing their robustness with mutation operations. Additionally, we compare them against cross-language models (C4, CLCDSA) known for detecting semantic clones. While single-language models show high F1 scores for BigCloneBench, their performance on SemanticCloneBench varies (up to 20%). Interestingly, the cross-language model (C4) shows superior performance (around 7%) on SemanticCloneBench over other models and performs similarly on BigCloneBench and GPTCloneBench. On mutation-based datasets, C4 has more robust performance (less than 1% difference) compared to single-language models, which show high variability. Accepted at the 40th IEEE International Conference on Software Maintenance and Evolution (ICSME 2024) |
| Document Type: | Article |
| DOI: | 10.1109/icsme58944.2024.00053 |
| DOI: | 10.48550/arxiv.2412.14739 |
| Access URL: | http://arxiv.org/abs/2412.14739 |
| Rights: | STM Policy #29 arXiv Non-Exclusive Distribution |
| Accession Number: | edsair.doi.dedup.....5c5d5db40038a642798ed292041c75d2 |
| Database: | OpenAIRE |
Nájsť tento článok vo Web of Science | Abstract: | Detecting and tracking code clones can ease various software development and maintenance tasks when changes in a code fragment should be propagated over all its copies. Several deep learning-based clone detection models have appeared in the literature for detecting syntactic and semantic clones, widely evaluated with the BigCloneBench dataset. However, class imbalance and the small number of semantic clones make BigCloneBench less ideal for interpreting model performance. Researchers also use other datasets such as GoogleCodeJam, OJClone, and SemanticCloneBench to understand model generalizability. To overcome the limitations of existing datasets, the GPT-assisted semantic and cross-language clone dataset GPTCloneBench has been released. However, how these models compare across datasets remains unclear. In this paper, we propose a multi-step evaluation approach for five state-of-the-art clone detection models leveraging existing benchmark datasets, including GPTCloneBench, and using mutation operators to study model ability. Specifically, we examine three highly-performing single-language models (ASTNN, GMN, CodeBERT) on BigCloneBench, SemanticCloneBench, and GPTCloneBench, testing their robustness with mutation operations. Additionally, we compare them against cross-language models (C4, CLCDSA) known for detecting semantic clones. While single-language models show high F1 scores for BigCloneBench, their performance on SemanticCloneBench varies (up to 20%). Interestingly, the cross-language model (C4) shows superior performance (around 7%) on SemanticCloneBench over other models and performs similarly on BigCloneBench and GPTCloneBench. On mutation-based datasets, C4 has more robust performance (less than 1% difference) compared to single-language models, which show high variability.<br />Accepted at the 40th IEEE International Conference on Software Maintenance and Evolution (ICSME 2024) |
|---|---|
| DOI: | 10.1109/icsme58944.2024.00053 |