Software Testing With Large Language Models: Survey, Landscape, and Vision

Pre-trained large language models (LLMs) have recently emerged as a breakthrough technology in natural language processing and artificial intelligence, with the ability to handle large-scale datasets and exhibit remarkable performance across a wide range of tasks. Meanwhile, software testing is a cr...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transactions on software engineering Ročník 50; číslo 4; s. 911 - 936
Hlavní autoři: Wang, Junjie, Huang, Yuchao, Chen, Chunyang, Liu, Zhe, Wang, Song, Wang, Qing
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York IEEE 01.04.2024
IEEE Computer Society
Témata:
Artificial intelligence
Codes
Computational modeling
GPT
Large language models
LLM
Natural language processing
Pre-trained large language model
Reviews
Software reliability
Software systems
Software testing
System effectiveness
Task analysis
ISSN:0098-5589, 1939-3520
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Pre-trained large language models (LLMs) have recently emerged as a breakthrough technology in natural language processing and artificial intelligence, with the ability to handle large-scale datasets and exhibit remarkable performance across a wide range of tasks. Meanwhile, software testing is a crucial undertaking that serves as a cornerstone for ensuring the quality and reliability of software products. As the scope and complexity of software systems continue to grow, the need for more effective software testing techniques becomes increasingly urgent, making it an area ripe for innovative approaches such as the use of LLMs. This paper provides a comprehensive review of the utilization of LLMs in software testing. It analyzes 102 relevant studies that have used LLMs for software testing, from both the software testing and LLMs perspectives. The paper presents a detailed discussion of the software testing tasks for which LLMs are commonly used, among which test case preparation and program repair are the most representative. It also analyzes the commonly used LLMs, the types of prompt engineering that are employed, as well as the accompanied techniques with these LLMs. It also summarizes the key challenges and potential opportunities in this direction. This work can serve as a roadmap for future research in this area, highlighting potential avenues for exploration, and identifying gaps in our current understanding of the use of LLMs in software testing.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0098-5589
1939-3520
DOI:10.1109/TSE.2024.3368208