Evaluating uniXcoder Embeddings for Automated Grading: A Study Across Varied Code Perspectives

Automated grading systems play a pivotal role in modern educational settings, offering efficiency and scalability in assessing student submissions. In this study, we investigate the potential of uniXcoder embeddings of C language codes. A comparative analysis across four scenarios: code-only, codequ...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:International Conference on Computing, Communication, and Networking Technologies (Online) S. 1 - 5
Hauptverfasser: Narmada, Nakka, Pati, Peeta Basa
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 24.06.2024
Schlagworte:
Automatic assessment
C languages
Code Embeddings
Codes
Davies Bouldin Index
Grading
Indexes
KL divergence
kmeans clustering
Principal component analysis
Principal Component Analysis (PCA)
Scalability
Silhouette Score
Subject matter experts
uniXcoder
Visualization
ISSN:2473-7674
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Automated grading systems play a pivotal role in modern educational settings, offering efficiency and scalability in assessing student submissions. In this study, we investigate the potential of uniXcoder embeddings of C language codes. A comparative analysis across four scenarios: code-only, codequestion, code-question-comment, and code-question-comment-reference is conducted. The dataset deals with C programs that are evaluated by Subject Matter Experts (SMEs). This study explores the analysis of code embeddings using K-Means approach and to determine the influence of Principal Component Analysis (PCA) and scaling techniques. The study compares clustering outcomes with and without PCA, with and without scaling techniques looking at measures including silhouette score, Davies-Bouldin index, Kullback-Leibler divergence and visualization techniques to draw conclusions. The findings suggest that uniXcoder embeddings demonstrate that "code-question-comment-reference" scenario is best suited for automated grading tasks. It also looks into the ability of the embedding to capture the level of error present in the code. This research contributes to the advancement of automated grading systems by providing insights into the suitability of uniXcoder embeddings for assessing student code submissions in educational environments.
ISSN:2473-7674
DOI:10.1109/ICCCNT61001.2024.10725702