A Study on How to Classify Exercises in Introductory Programming Courses: A Moodle Plugin Contribution - Versissimo, Thiago Gomes; De Oliveira Brandao, Leonidas; Haar, Ewout ter (Proceedings - Frontiers in Education Conference, 2023)
This full paper reports the results of the development of a Moodle plugin, Programming Exercise Teaching Assistant (PETA), created to support teachers from introductory programming courses to identify problematic exercises by analyzing how students interact with them. Moodle is an open source learning management system that allows personalization of its functionality through plugins. In introductory programming courses, it is common to propose problems to students, for which they need to elaborate solution code. The VPL (Virtual Programming Lab) is a popular plugin that allows teachers to create this kind of problem, with automatic evaluation built on top of test-cases. The iAssign (interactive Assignment) is another plugin of the same sort, but with block-based programming. Our plugin is an open-source project, initially developed to integrate with iAssign, and created to integrate easily with any similar plugin used in programming tasks. Our primary goal is to empower teachers and students by supplying a tool that improves the quality of programming in education. Over the last few years, the University of São Paulo has increased its public policy for social inclusion, bringing in even more learners with diverse educational backgrounds. Understanding the students' profiles and being able to modify problems based on these new contexts is fundamental to improve the quality of teaching. This plugin aims to meet the needs of teachers and learners, exploring data from their educational context to improve the learning experience. The plugin automatically extracts data from students' interaction with the coding exercise, e.g. the time between submission (TBS), code changing between submissions (CCBS), and grades. Using these variables, we calculated derived metrics in an attempt to classify exercises according to different parameters. First, we considered time-related metrics, the highest time between submissions (HT). Secondly, we evaluated metrics related to code changes, the highest code changes between submissions (HC). Finally, we accounted for student performance by considering average grades, penalized by time and by number of submissions (AG). A technical contribution of this work is the Moodle plugin that analyzes the students' codes of iAssign, available as free software. The results from this new plugin with eight introductory classes show that it was possible to classify exercises. The AG metric created could be associated with the exercise level of difficulty, while HT and HC metrics are indicative of exercise complexity, requiring a higher level of prior knowledge, such as familiarity with loops, functions, and potential challenges with the problem instructions.