Development of an Automated Scoring Model Using SentenceTransformers for Discussion Forums in Online Learning Environments

Due to the limitations of public datasets, research on automatic essay scoring in Indonesian has been restrained and resulted in suboptimal accuracy. In general, the main goal of the essay scoring system is to improve execution time, which is usually done manually with human judgment. This study use...

Full description

Saved in:
Bibliographic Details
Published in:Journal of computing and information technology Vol. 30; no. 2; pp. 85 - 99
Main Authors: Dhini, Bachriah Fatwa, Girsang, Abba Suganda
Format: Journal Article Paper
Language:English
Published: Sveuciliste U Zagrebu 01.06.2022
Fakultet elektrotehnike i računarstva Sveučilišta u Zagrebu
Subjects:
Analysis
Automatic Essay Scoring, Discussion Forum, SentenceTransformers, Monolingual Model, Multilingual Model
Computational linguistics
Language processing
Learning management systems
Natural language interfaces
Online education
Rubrics (Education)
School prose
Technology application
Indonesia
ISSN:1330-1136, 1846-3908
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Due to the limitations of public datasets, research on automatic essay scoring in Indonesian has been restrained and resulted in suboptimal accuracy. In general, the main goal of the essay scoring system is to improve execution time, which is usually done manually with human judgment. This study uses a discussion forum in online learning to generate an assessment between the responses and the lecturer's rubric in the automated essay scoring. A SentenceTransformers pre-trained model that can construct the highest vector embedding was proposed to identify the semantic meaning between the responses and the lecturer's rubric. The effectiveness of monolingual and multilingual models was compared. This research aims to determine the model's effectiveness and the appropriate model for the Automated Essay Scoring (AES) used in paired sentence Natural Language Processing tasks. The distiluse-base-multilingual-cased-v1 model, which employed the Pearson correlation method, obtained the highest performance. Specifically, it obtained a correlation value of 0.63 and a mean absolute error (MAE) score of 0.70. It indicates that the overall prediction result is enhanced when compared to the earlier regression task research.
AbstractList Due to the limitations of public datasets, research on automatic essay scoring in Indonesian has been restrained and resulted in suboptimal accuracy. In general, the main goal of the essay scoring system is to improve execution time, which is usually done manually with human judgment. This study uses a discussion forum in online learning to generate an assessment between the responses and the lecturer's rubric in the automated essay scoring. A SentenceTransformers pre-trained model that can construct the highest vector embedding was proposed to identify the semantic meaning between the responses and the lecturer's rubric. The effectiveness of monolingual and multilingual models was compared. This research aims to determine the model's effectiveness and the appropriate model for the Automated Essay Scoring (AES) used in paired sentence Natural Language Processing tasks. The distiluse-base-multilingual-cased-v1 model, which employed the Pearson correlation method, obtained the highest performance. Specifically, it obtained a correlation value of 0.63 and a mean absolute error (MAE) score of 0.70. It indicates that the overall prediction result is enhanced when compared to the earlier regression task research. ACM CCS (2012) Classification: Computing methodologies [right arrow] Modeling and simulation [right arrow] Model development and analysis [right arrow] Model verification and validation Applied computing [right arrow] Education [right arrow] Distance learning Keywords: Automatic Essay Scoring, Discussion Forum, SentenceTransformers, Monolingual Model, Multilingual Model
Due to the limitations of public datasets, research on automatic essay scoring in Indonesian has been restrained and resulted in suboptimal accuracy. In general, the main goal of the essay scoring system is to improve execution time, which is usually done manually with human judgment. This study uses a discussion forum in online learning to generate an assessment between the responses and the lecturer's rubric in the automated essay scoring. A SentenceTransformers pre-trained model that can construct the highest vector embedding was proposed to identify the semantic meaning between the responses and the lecturer's rubric. The effectiveness of monolingual and multilingual models was compared. This research aims to determine the model's effectiveness and the appropriate model for the Automated Essay Scoring (AES) used in paired sentence Natural Language Processing tasks. The distiluse-base-multilingual-cased-v1 model, which employed the Pearson correlation method, obtained the highest performance. Specifically, it obtained a correlation value of 0.63 and a mean absolute error (MAE) score of 0.70. It indicates that the overall prediction result is enhanced when compared to the earlier regression task research.
Audience Academic
Author Dhini, Bachriah Fatwa
Girsang, Abba Suganda
Author_xml – sequence: 1
  givenname: Bachriah Fatwa
  surname: Dhini
  fullname: Dhini, Bachriah Fatwa
  organization: Bina Nusantara University, Jakarta, Indonesia
– sequence: 2
  givenname: Abba Suganda
  surname: Girsang
  fullname: Girsang, Abba Suganda
  organization: Bina Nusantara University, Jakarta, Indonesia
BookMark eNp1kU9v1DAQxSNUJErpB-BmiROHLHYmTpzjqn-g0qJKbHu2HGe8GBK7srNV4dMzYcthEdgHP41-z6OZ97o4CTFgUbwVfFVxCdUH62dSVbUSnMu6VS-KU6HqpoSOqxPSALwUAppXxXnO3zgd6JqmFqfFz0t8xDE-TBhmFh0zga33c5zMjAPb2ph82LHPccCR3edFbwnEYPEumZBdTBOmzOhllz7bfc4-BnYd037KzAd2G0YfkG3QpLC4r8KjTzEs3fKb4qUzY8bz5_esuL--urv4VG5uP95crDelhbpSZecU1q4bOEqUwrqhAwlqAJC8EtA7aXtQ0rqm762TvcOFxhZM2yvV8wHOivLw79dkzXf9kPxk0g8djdeHSk4WSWrgXcUV8e8O_M6MqH1wcU7GTjSeXreyg7ZrGyBq9Q-K7oCTt5SP81Q_Mrw_MhAz49O8M7Q0fbP9csy2B9ammHNCpylhM9NuqYkfteD6d_BLWS_B6-fgySn-cv6Z9_-eX2IdtFg
CODEN CJCTEM
CitedBy_id crossref_primary_10_1080_09540091_2025_2518991
crossref_primary_10_1108_AAOUJ_02_2023_0027
ContentType Journal Article
Paper
Copyright COPYRIGHT 2022 Sveuciliste U Zagrebu
Copyright_xml – notice: COPYRIGHT 2022 Sveuciliste U Zagrebu
DBID AAYXX
CITATION
ISR
VP8
DOI 10.20532/cit.2022.1005478
DatabaseName CrossRef
Gale In Context: Science
Portal of Croatian Scientific and Professional Journals – HRČAK
DatabaseTitle CrossRef
DatabaseTitleList

CrossRef
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISSN 1846-3908
EndPage 99
ExternalDocumentID oai_hrcak_srce_hr_309208
A759379763
10_20532_cit_2022_1005478
GeographicLocations Indonesia
GeographicLocations_xml – name: Indonesia
GroupedDBID .4S
.DC
29B
29K
2WC
5GY
5VS
77I
AAYXX
ADMLS
ALMA_UNASSIGNED_HOLDINGS
ARCSS
BAIFH
BBTPI
CITATION
CS3
D-I
DU5
E3Z
EBS
EDO
EJD
EN8
EOJEC
GROUPED_DOAJ
I-F
IAO
ICD
ISR
ITC
IVC
KQ8
KWQ
MK~
ML~
M~E
OBODZ
OK1
OVT
P2P
PV9
RZL
TR2
TUS
VP8
XH6
ICW
IPNFZ
RIG
ID FETCH-LOGICAL-c3428-9f8e4f9d0e5e51cfd93538d3350213bf5cb385cf6bbcf5bfee4f9e73a7b88b0d3
ISSN 1330-1136
IngestDate Fri Oct 27 04:15:06 EDT 2023
Tue Nov 11 10:41:42 EST 2025
Sat Nov 29 11:07:07 EST 2025
Wed Nov 26 11:31:11 EST 2025
Sat Nov 29 04:13:47 EST 2025
Tue Nov 18 22:29:52 EST 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue 2
Language English
License cc-by-nd: openAccess
LinkModel OpenURL
MergedId FETCHMERGED-LOGICAL-c3428-9f8e4f9d0e5e51cfd93538d3350213bf5cb385cf6bbcf5bfee4f9e73a7b88b0d3
Notes 309208
OpenAccessLink http://dx.doi.org/10.20532/cit.2022.1005478
PageCount 15
ParticipantIDs hrcak_primary_oai_hrcak_srce_hr_309208
gale_infotracmisc_A759379763
gale_infotracacademiconefile_A759379763
gale_incontextgauss_ISR_A759379763
crossref_citationtrail_10_20532_cit_2022_1005478
crossref_primary_10_20532_cit_2022_1005478
PublicationCentury 2000
PublicationDate 20220601
PublicationDateYYYYMMDD 2022-06-01
PublicationDate_xml – month: 06
  year: 2022
  text: 20220601
  day: 01
PublicationDecade 2020
PublicationTitle Journal of computing and information technology
PublicationYear 2022
Publisher Sveuciliste U Zagrebu
Fakultet elektrotehnike i računarstva Sveučilišta u Zagrebu
Publisher_xml – name: Sveuciliste U Zagrebu
– name: Fakultet elektrotehnike i računarstva Sveučilišta u Zagrebu
SSID ssj0000396641
Score 2.2129784
Snippet Due to the limitations of public datasets, research on automatic essay scoring in Indonesian has been restrained and resulted in suboptimal accuracy. In...
SourceID hrcak
gale
crossref
SourceType Open Access Repository
Aggregation Database
Enrichment Source
Index Database
StartPage 85
SubjectTerms Analysis
Automatic Essay Scoring, Discussion Forum, SentenceTransformers, Monolingual Model, Multilingual Model
Computational linguistics
Language processing
Learning management systems
Natural language interfaces
Online education
Rubrics (Education)
School prose
Technology application
Title Development of an Automated Scoring Model Using SentenceTransformers for Discussion Forums in Online Learning Environments
URI https://hrcak.srce.hr/309208
Volume 30
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVAON
  databaseName: DOAJ Directory of Open Access Journals
  customDbUrl:
  eissn: 1846-3908
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0000396641
  issn: 1330-1136
  databaseCode: DOA
  dateStart: 20160101
  isFulltext: true
  titleUrlDefault: https://www.doaj.org/
  providerName: Directory of Open Access Journals
– providerCode: PRVHPJ
  databaseName: ROAD: Directory of Open Access Scholarly Resources
  customDbUrl:
  eissn: 1846-3908
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0000396641
  issn: 1330-1136
  databaseCode: M~E
  dateStart: 20000101
  isFulltext: true
  titleUrlDefault: https://road.issn.org
  providerName: ISSN International Centre
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1Lj9MwELbKLgcuLE9RdllZCIFEFJHm0SRHYLeABCtEu9KKS2RPnTaiSlfpgxUH_gv_lBnbSdPloeXAJU1dt3Yzn2fG43kw9gQlXIBsLnQFeOCGuIJcISLcpaiQQnyUJxMdKPw-PjlJzs7Sj53OjzoWZj2LyzK5uEjP_yupsQ2JTaGz_0Du5kexAe-R6HhFsuP1SoRvuQHpQ36yZyznqJiSagnG344qoM0c4y4wpKycSPpRrcJSSC85Hx4VC1iRk2zpDChlg_acNalJ67SsE-e4FSj3B0UXdOGIOhjSZmrVsFv-Ytc_mha6xpTzSsAUH9bUGYjl10Z0DFcTsnxohialcN4U1UJY2WtNF7jrbVysjOlyrVZQzAjOzqnzWUwqJVctXhwEnkslZ9rM2h7iFK09s-G8pvDPZYHgU-ELpBgU5Dfr--QUQhnMNtKvPvF_N_y03WiSAscR6m8E6Gts18c35DX44fvGjOcFuF8MzbbeTtecneuRX1wed0v7sTrA7rQC8aWl1YxusZuWSvylgdFt1lHlHbZXl_rglvPfZd9aqOLznIuSN6jiFlVco4prVPHfoYrjK9-gihtU8aLkBlW8RhVvo-oeOx0cj16_dW3ZDhdw2aP4zBMV5unYU5GKepCP0wCl6hhXPuqTgcwjkEESQd6XEvJI5op6qzgQsUwS6Y2D-2ynnJfqAeMgo7DfA78vRBymkCZRGgIqsTKRfk_20y7z6seZgc1pT6VVZhnubTUFqDkjCmSWAl32vPnKuUno8rfOj4lGGSVKKckTayLwCWWIlWyDjC57ZjvlcxwchA1swb9AudW2eh5s9URODlsfP9VQaOZFqd9Ny6IChbdZ4KW-lzy8yrT22Y3NmjtgO8tqpR6x67BeFovqUNueDjWWfwLoctFJ
linkProvider ISSN International Centre
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Development+of+an+Automated+Scoring+Model+Using+SentenceTransformers+for+Discussion+Forums+in+Online+Learning+Environments&rft.jtitle=Journal+of+computing+and+information+technology&rft.au=Dhini%2C+Bachriah+Fatwa&rft.au=Suganda%2C+Abba+Girsang&rft.date=2022-06-01&rft.pub=Sveuciliste+U+Zagrebu&rft.issn=1330-1136&rft.volume=30&rft.issue=2&rft.spage=85&rft_id=info:doi/10.20532%2Fcit.2022.1005478&rft.externalDBID=ISR&rft.externalDocID=A759379763
thumbnail_s http://cvtisr.summon.serialssolutions.com/2.0.0/image/custom?url=https%3A%2F%2Fhrcak.srce.hr%2Flogo_broj%2F23904.jpg