Optimizing Large Language Models for Auto-Generation of Programming Quizzes

This study analyzes the use of Large Language Models (LLMs) like ChatGPT in creating quizzes for Java programming courses, specifically Object-Oriented Programming (CS1) and Data Structures (CS2). It aims to evaluate the accuracy of LLM-generated assessments, understand the benefits and drawbacks of...

Full description

Saved in:
Bibliographic Details
Published in:Integrated STEM Education Conference (Online) pp. 1 - 5
Main Authors: Kumar, Yulia, Manikandan, Anjana, Li, J. Jenny, Morreale, Patricia
Format: Conference Proceeding
Language:English
Published: IEEE 09.03.2024
Subjects:
Accuracy
AI-Supplemental Instructor (AI-SI)
Chatbots
Computer science education
Data structures
Education
Java
Java programming instruction
Large language models
use of LLMs in CS education
ISSN:2473-7623
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This study analyzes the use of Large Language Models (LLMs) like ChatGPT in creating quizzes for Java programming courses, specifically Object-Oriented Programming (CS1) and Data Structures (CS2). It aims to evaluate the accuracy of LLM-generated assessments, understand the benefits and drawbacks of using LLMs in CS education from educators' viewpoints, and identify effective prompt engineering strategies to enhance the quality of educational materials. The research compares quizzes made by LLMs against human-created content to assess their consistency with Java programming principles, alignment with CS1 and CS2 learning goals, and their impact on student engagement and comprehension, providing insights into LLMs' effectiveness in academic assessment creation for computer science education.
ISSN:2473-7623
DOI:10.1109/ISEC61299.2024.10665141