A Survey Study on the State of the Art of Programming Exercise Generation Using Large Language Models

This paper analyzes Large Language Models (LLMs) with regard to their programming exercise generation capabilities. Through a survey study, we defined the state of the art, extracted their strengths and weaknesses and finally proposed an evaluation matrix, helping researchers and educators to decide...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Proceedings / Conference on Software Engineering Education & Training S. 1 - 5
Hauptverfasser: Frankford, Eduard, Hohn, Ingo, Sauerwein, Clemens, Breu, Ruth
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 29.07.2024
Schlagworte:
Artificial Intelligence
ChatGPT
Data privacy
Education
Fitting
Large language models
Navigation
Programming Education
Programming Exercise Generation
Programming Exercise Generation Benchmark
Surveys
Transforms
ISSN:2377-570X
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:This paper analyzes Large Language Models (LLMs) with regard to their programming exercise generation capabilities. Through a survey study, we defined the state of the art, extracted their strengths and weaknesses and finally proposed an evaluation matrix, helping researchers and educators to decide which LLM is the best fitting for the programming exercise generation use case. We also found that multiple LLMs are capable of producing useful program-ming exercises. Nevertheless, there exist challenges like the ease with which LLMs might solve exercises generated by LLMs. This paper contributes to the ongoing discourse on the integration of LLMs in education.
ISSN:2377-570X
DOI:10.1109/CSEET62301.2024.10662990