A High-Quality Generation Approach for Educational Programming Projects Using LLM

High-quality programming projects for education are critically required in teaching. However, it is hard to develop those projects efficiently and artificially constrained by the lecturers' experience and background. The recent popularity of large language models (LLMs) has led to a great numbe...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transactions on learning technologies Ročník 17; s. 2242 - 2255
Hlavní autoři: Song, Tian, Zhang, Hang, Xiao, Yijia
Médium: Journal Article
Jazyk:angličtina
Vydáno: IEEE 2024
Témata:
Automatic generation
Chatbots
Codes
Computer architecture
Education
generative pretrained transformer (GPT)
Iterative methods
large language models (LLMs)
Logic
Measurement
programming education
Programming profession
programming projects
Prompt engineering
Python
ISSN:1939-1382, 2372-0050
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:High-quality programming projects for education are critically required in teaching. However, it is hard to develop those projects efficiently and artificially constrained by the lecturers' experience and background. The recent popularity of large language models (LLMs) has led to a great number of applications in the field of education, but concerns persist that the output might be unreliable when dealing with intricate requirements. In this study, we design a customized role-based agent (CRBA), which can be configured for different roles specializing in specific areas of expertise, making the LLM yield content of higher specialization. An iterative architecture of multi-CRBAs is proposed to generate multistep projects, where CRBAs automatically criticize and optimize the LLM's intermediate outputs to enhance quality. We propose ten evaluation metrics across three aspects to assess project quality through expert grading. Further, we conduct an A/B test among 60 undergraduate students in a programming course and collect their feedback through a questionnaire. According to the students' rating results, the LLM-generated projects have comparable performance to man-made ones in terms of project description, learning step setting, assistance to students, and overall project quality. This study effectively integrates LLM into educational scenarios and enhances the efficiency of creating high-quality and practical programming exercises for lecturers.
ISSN:1939-1382
2372-0050
DOI:10.1109/TLT.2024.3499751