Semantic Analysis of ChatGPT's Behavior in Contextual Code Correction for Python.
Saved in:
| Title: | Semantic Analysis of ChatGPT's Behavior in Contextual Code Correction for Python. |
|---|---|
| Authors: | Alhalangy, Abdalilah |
| Source: | International Journal of Computational & Experimental Science & Engineering Experimental Science & Engineering (IJCESEN); 2025, Vol. 11 Issue 3, p5036-5042, 7p |
| Subject Terms: | CHATGPT, PYTHON programming language, DEBUGGING, COMPUTER software development, LANGUAGE models |
| Abstract: | Software debugging remains one of the most time-consuming and cognitively demanding phases in the software development lifecycle. The paper considers the analytical capabilities of transformers transformer-based language models--specifically ChatGPT--in detecting and correcting semantic faults in multi-module Python projects. Ten synthetic Python programs (100-200 lines each), containing a total of 30 deliberately injected faults (distributed among order faults, variable leakage, and edgecase omissions), were submitted to ChatGPT under a standardized prompting scheme. Model responses were benchmarked against conventional static analysis tools (Pylint, MyPy) and a human expert baseline. Quantitatively, ChatGPT achieved an average error detection rate (EDR) of 76.7%, outperforming Pylint (23.3%) and MyPy (15%) across fault categories. In repair accuracy (RA), ChatGPT resolved 62-75% of the identified errors correctly, versus 10-25% for static tools. Statistical validation using a Chi-square test (² = 36.27, p < 0.001) and one-way ANOVA (F(3, 27) = 14.92, p < 0.001) confirms the significance of these differences. Qualitative clarity was also assessed using ordinal metrics and validated via Kruskal-Wallis H tests (H = 11.56, p < 0.01). These results suggest that ChatGPT possesses substantial semantic reasoning capabilities, particularly in contexts requiring non-local inference across modules. However, limitations persist in handling implicit dependencies and dynamic runtime conditions. The study concludes that such models can be meaningfully integrated into debugging pipelines as assistive agents, provided their outputs are cross-validated with expert oversight and static tools. Future research should explore hybrid frameworks that combine statistical inference with formal verification techniques. [ABSTRACT FROM AUTHOR] |
| Copyright of International Journal of Computational & Experimental Science & Engineering Experimental Science & Engineering (IJCESEN) is the property of Journal of Computational Experimental Science, Engineering Experimental Science & Engineering and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Complementary Index |
Nájsť tento článok vo Web of Science Be the first to leave a comment!