A Survey Study on the State of the Art of Programming Exercise Generation Using Large Language Models

This paper analyzes Large Language Models (LLMs) with regard to their programming exercise generation capabilities. Through a survey study, we defined the state of the art, extracted their strengths and weaknesses and finally proposed an evaluation matrix, helping researchers and educators to decide...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings / Conference on Software Engineering Education & Training s. 1 - 5
Hlavní autoři: Frankford, Eduard, Hohn, Ingo, Sauerwein, Clemens, Breu, Ruth
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 29.07.2024
Témata:
Artificial Intelligence
ChatGPT
Data privacy
Education
Fitting
Large language models
Navigation
Programming Education
Programming Exercise Generation
Programming Exercise Generation Benchmark
Surveys
Transforms
ISSN:2377-570X
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:This paper analyzes Large Language Models (LLMs) with regard to their programming exercise generation capabilities. Through a survey study, we defined the state of the art, extracted their strengths and weaknesses and finally proposed an evaluation matrix, helping researchers and educators to decide which LLM is the best fitting for the programming exercise generation use case. We also found that multiple LLMs are capable of producing useful program-ming exercises. Nevertheless, there exist challenges like the ease with which LLMs might solve exercises generated by LLMs. This paper contributes to the ongoing discourse on the integration of LLMs in education.
ISSN:2377-570X
DOI:10.1109/CSEET62301.2024.10662990