Detection of Malicious Codes Generated by Large Language Models: A Comparison of GPT-3.5, GPT-4o, Gemini, and Claude.
Uloženo v:
| Název: | Detection of Malicious Codes Generated by Large Language Models: A Comparison of GPT-3.5, GPT-4o, Gemini, and Claude. |
|---|---|
| Autoři: | Pehlivanoğlu, Meltem Kurt1 meltem.kurt@kocaeli.edu.tr, Çoban, Murat Görkem1 |
| Zdroj: | International Journal of Information Security Science. 2025, Vol. 14 Issue 1, p1-12. 12p. |
| Témata: | *LANGUAGE models, *MACHINE learning, *ARTIFICIAL intelligence, *GENERATIVE pre-trained transformers, *RANDOM forest algorithms |
| Abstrakt: | This study presents novel machine learning-based approaches for detecting whether source code generated by Large Language Models (LLMs) contains malicious code. To achieve this, comprehensive datasets comprising malicious and benign code samples were created using the GPT-3.5 (ChatGPT), GPT-4o, Gemini, and Claude language models. The extracted code samples were then processed through CodeBERT, CodeT5, and manual feature extraction techniques before being classified using various machine learning algorithms. Experimental results demonstrate that this approach can effectively detect malicious software in code generated by LLMs. This study makes contributions to software security and represents a crucial step toward preventing the misuse of LLMs for malicious purposes. Moreover, the Random Forest algorithm for binary malicious code classification in LLM-generated code achieved the best F1 score of 94.92% on the ChatGPT-generated dataset (with CodeT5 feature extraction technique). We also showed that the classification models exhibited poor performance on the dataset generated by Claude language model. [ABSTRACT FROM AUTHOR] |
| Databáze: | Academic Search Index |
Nájsť tento článok vo Web of Science Buďte první, kdo okomentuje tento záznam!