Computer Science Education in ChatGPT Era: Experiences from an Experiment in a Programming Course for Novice Programmers

The use of large language models with chatbots like ChatGPT has become increasingly popular among students, especially in Computer Science education. However, significant debates exist in the education community on the role of ChatGPT in learning. Therefore, it is critical to understand the potentia...

Full description

Saved in:
Bibliographic Details
Published in:Mathematics (Basel) Vol. 12; no. 5; p. 629
Main Authors: Kosar, Tomaž, Ostojić, Dragana, Liu, Yu David, Mernik, Marjan
Format: Journal Article
Language:English
Published: Basel MDPI AG 01.03.2024
Subjects:
Artificial intelligence
Chatbots
ChatGPT
Cheating
Computational linguistics
Computer science
Computers
controlled experiment
Critical thinking
Distance learning
Education
Experiments
Language processing
Large language models
Natural language interfaces
Object oriented programming
Programmers
Science education
Sciences education
Software
software engineering education
Students
Verbal communication
ISSN:2227-7390, 2227-7390
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The use of large language models with chatbots like ChatGPT has become increasingly popular among students, especially in Computer Science education. However, significant debates exist in the education community on the role of ChatGPT in learning. Therefore, it is critical to understand the potential impact of ChatGPT on the learning, engagement, and overall success of students in classrooms. In this empirical study, we report on a controlled experiment with 182 participants in a first-year undergraduate course on object-oriented programming. Our differential study divided students into two groups, one using ChatGPT and the other not using it for practical programming assignments. The study results showed that the students’ performance is not influenced by ChatGPT usage (no statistical significance between groups with a p-value of 0.730), nor are the grading results of practical assignments (p-value 0.760) and midterm exams (p-value 0.856). Our findings from the controlled experiment suggest that it is safe for novice programmers to use ChatGPT if specific measures and adjustments are adopted in the education process.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2227-7390
2227-7390
DOI:10.3390/math12050629