Evaluating uniXcoder Embeddings for Automated Grading: A Study Across Varied Code Perspectives

Automated grading systems play a pivotal role in modern educational settings, offering efficiency and scalability in assessing student submissions. In this study, we investigate the potential of uniXcoder embeddings of C language codes. A comparative analysis across four scenarios: code-only, codequ...

Full description

Saved in:
Bibliographic Details
Published in:International Conference on Computing, Communication, and Networking Technologies (Online) pp. 1 - 5
Main Authors: Narmada, Nakka, Pati, Peeta Basa
Format: Conference Proceeding
Language:English
Published: IEEE 24.06.2024
Subjects:
Automatic assessment
C languages
Code Embeddings
Codes
Davies Bouldin Index
Grading
Indexes
KL divergence
kmeans clustering
Principal component analysis
Principal Component Analysis (PCA)
Scalability
Silhouette Score
Subject matter experts
uniXcoder
Visualization
ISSN:2473-7674
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Automated grading systems play a pivotal role in modern educational settings, offering efficiency and scalability in assessing student submissions. In this study, we investigate the potential of uniXcoder embeddings of C language codes. A comparative analysis across four scenarios: code-only, codequestion, code-question-comment, and code-question-comment-reference is conducted. The dataset deals with C programs that are evaluated by Subject Matter Experts (SMEs). This study explores the analysis of code embeddings using K-Means approach and to determine the influence of Principal Component Analysis (PCA) and scaling techniques. The study compares clustering outcomes with and without PCA, with and without scaling techniques looking at measures including silhouette score, Davies-Bouldin index, Kullback-Leibler divergence and visualization techniques to draw conclusions. The findings suggest that uniXcoder embeddings demonstrate that "code-question-comment-reference" scenario is best suited for automated grading tasks. It also looks into the ability of the embedding to capture the level of error present in the code. This research contributes to the advancement of automated grading systems by providing insights into the suitability of uniXcoder embeddings for assessing student code submissions in educational environments.
ISSN:2473-7674
DOI:10.1109/ICCCNT61001.2024.10725702