In EDS ansehen

SQL Autograder: Web-based LLM-powered Autograder for Assessment of SQL Queries.

Gespeichert in:
Bibliographische Detailangaben
Titel: SQL Autograder: Web-based LLM-powered Autograder for Assessment of SQL Queries.
Autoren: Manikani, Karan, Chapaneri, Radhika, Shetty, Dharmik, Shah, Divyata
Quelle: International Journal of Artificial Intelligence in Education (Springer Science & Business Media B.V.); Dec2025, Vol. 35 Issue 4, p2047-2077, 31p
Schlagwörter: SQL, LANGUAGE models, EDUCATIONAL technology, DATABASES, WEB-based user interfaces, EDUCATIONAL evaluation
Abstract: Structured query language (SQL) queries are an important aspect of database concepts in the information technology (IT) domain. Evaluation of SQL queries ensures that the learners can understand and apply various SQL concepts correctly. However, this can be a laborious task when carried out manually by course instructors at universities, which often does not scale well. To address these limitations, this study proposes a web-based application, SQL autograder, which can be used by instructors of a university course to evaluate assessments and enhance the quality of education and learning outcomes. We propose a framework that makes use of large language models (LLMs) to assess the correctness of SQL queries submitted by students. We train a variety of open-source LLMs of varying sizes on a diverse dataset of SQL queries, with queries ranging from simple ones that include a single JOIN statement to more complex ones involving multiple SQL features. We implemented and tested our LLM-based framework in real-world educational settings for a university course, which shows promising results in enhancing the learning experience for students by providing instant feedback on areas needing improvement. We tested our application on 88 participants and found that the autograder is 180x faster than the instructor, with an average accuracy of 96.77%. After taking the qualitative feedback from the participants, 97% of them found it to be useful. The proposed framework reduces the workload of instructors by offering a more scalable and consistent evaluation process that enhances the performance of students. [ABSTRACT FROM AUTHOR]
Copyright of International Journal of Artificial Intelligence in Education (Springer Science & Business Media B.V.) is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Datenbank: Complementary Index
Beschreibung
Abstract:Structured query language (SQL) queries are an important aspect of database concepts in the information technology (IT) domain. Evaluation of SQL queries ensures that the learners can understand and apply various SQL concepts correctly. However, this can be a laborious task when carried out manually by course instructors at universities, which often does not scale well. To address these limitations, this study proposes a web-based application, SQL autograder, which can be used by instructors of a university course to evaluate assessments and enhance the quality of education and learning outcomes. We propose a framework that makes use of large language models (LLMs) to assess the correctness of SQL queries submitted by students. We train a variety of open-source LLMs of varying sizes on a diverse dataset of SQL queries, with queries ranging from simple ones that include a single JOIN statement to more complex ones involving multiple SQL features. We implemented and tested our LLM-based framework in real-world educational settings for a university course, which shows promising results in enhancing the learning experience for students by providing instant feedback on areas needing improvement. We tested our application on 88 participants and found that the autograder is 180x faster than the instructor, with an average accuracy of 96.77%. After taking the qualitative feedback from the participants, 97% of them found it to be useful. The proposed framework reduces the workload of instructors by offering a more scalable and consistent evaluation process that enhances the performance of students. [ABSTRACT FROM AUTHOR]
ISSN:15604292
DOI:10.1007/s40593-025-00460-2