Development of an Automated Scoring Model Using SentenceTransformers for Discussion Forums in Online Learning Environments

Due to the limitations of public datasets, research on automatic essay scoring in Indonesian has been restrained and resulted in suboptimal accuracy. In general, the main goal of the essay scoring system is to improve execution time, which is usually done manually with human judgment. This study use...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Journal of computing and information technology Ročník 30; číslo 2; s. 85 - 99
Hlavní autoři: Dhini, Bachriah Fatwa, Girsang, Abba Suganda
Médium: Journal Article Paper
Jazyk:angličtina
Vydáno: Sveuciliste U Zagrebu 01.06.2022
Fakultet elektrotehnike i računarstva Sveučilišta u Zagrebu
Témata:
Analysis
Automatic Essay Scoring, Discussion Forum, SentenceTransformers, Monolingual Model, Multilingual Model
Computational linguistics
Language processing
Learning management systems
Natural language interfaces
Online education
Rubrics (Education)
School prose
Technology application
Indonesia
ISSN:1330-1136, 1846-3908
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Due to the limitations of public datasets, research on automatic essay scoring in Indonesian has been restrained and resulted in suboptimal accuracy. In general, the main goal of the essay scoring system is to improve execution time, which is usually done manually with human judgment. This study uses a discussion forum in online learning to generate an assessment between the responses and the lecturer's rubric in the automated essay scoring. A SentenceTransformers pre-trained model that can construct the highest vector embedding was proposed to identify the semantic meaning between the responses and the lecturer's rubric. The effectiveness of monolingual and multilingual models was compared. This research aims to determine the model's effectiveness and the appropriate model for the Automated Essay Scoring (AES) used in paired sentence Natural Language Processing tasks. The distiluse-base-multilingual-cased-v1 model, which employed the Pearson correlation method, obtained the highest performance. Specifically, it obtained a correlation value of 0.63 and a mean absolute error (MAE) score of 0.70. It indicates that the overall prediction result is enhanced when compared to the earlier regression task research.
Bibliografie:309208
ISSN:1330-1136
1846-3908
DOI:10.20532/cit.2022.1005478