Constructed Response or Multiple-Choice Questions for Assessing Declarative Programming Knowledge? That is the Question

Aim/Purpose: This paper presents a data mining approach for analyzing responses to advanced declarative programming questions. The goal of this research is to find a model that can explain the results obtained by students when they perform exams with Constructed Response questions and with equivalen...

Full description

Saved in:
Bibliographic Details
Published in:Journal of Information Technology Education. Innovations in Practice Vol. 18; pp. 153 - 170
Main Authors: Belo, Yolanda, Moro, Sérgio, Martins, António, Ramos, Pedro, Martinho Costa, Joana, Esmerado, Joaquim
Format: Journal Article
Language:English
Published: Santa Rosa Informing Science Institute 2019
Subjects:
Educational Assessment
Educational Objectives
Feedback (Response)
High Achievement
Learning Analytics
Learning Processes
Measurement Techniques
Multiple choice
Outcomes of Education
Programming
Researchers
Taxonomy
ISSN:2165-3151, 2165-316X
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Aim/Purpose: This paper presents a data mining approach for analyzing responses to advanced declarative programming questions. The goal of this research is to find a model that can explain the results obtained by students when they perform exams with Constructed Response questions and with equivalent Multiple-Choice Questions. Background: The assessment of acquired knowledge is a fundamental role in the teaching-learning process. It helps to identify the factors that can contribute to the teacher in the developing of pedagogical methods and evaluation tools and it also contributes to the self-regulation process of learning. However, better format of questions to assess declarative programming knowledge is still a subject of ongoing debate. While some research advocates the use of constructed responses, others emphasize the potential of multiple-choice questions. Methodology: A sensitivity analysis was applied to extract useful knowledge from the relevance of the characteristics (i.e., the input variables) used for the data mining process to compute the score. Contribution: Such knowledge helps the teachers to decide which format they must consider with respect to the objectives and expected students results. Findings: The results shown a set of factors that influence the discrepancy between answers in both formats. Recommendations for Practitioners: Teachers can make an informed decision about whether to choose multiple-choice questions or constructed-response taking into account the results of this study. Recommendation for Researchers: In this study a block of exams with CR questions is verified to complement the area of learning, returning greater performance in the evaluation of students and improving the teaching-learning process. Impact on Society: The results of this research confirm the findings of several other researchers that the use of ICT and the application of MCQ is an added value in the evaluation process. In most cases the student is more likely to succeed with MCQ, however if the teacher prefers to evaluate with CR other research approaches are needed. Future Research: Future research must include other question formats.
AbstractList Aim/Purpose This paper presents a data mining approach for analyzing responses to advanced declarative programming questions. The goal of this research is to find a model that can explain the results obtained by students when they perform exams with Constructed Response questions and with equivalent Multiple-Choice Questions.
Aim/Purpose: This paper presents a data mining approach for analyzing responses to advanced declarative programming questions. The goal of this research is to find a model that can explain the results obtained by students when they perform exams with Constructed Response questions and with equivalent Multiple-Choice Questions. Background: The assessment of acquired knowledge is a fundamental role in the teaching-learning process. It helps to identify the factors that can contribute to the teacher in the developing of pedagogical methods and evaluation tools and it also contributes to the self-regulation process of learning. However, better format of questions to assess declarative programming knowledge is still a subject of ongoing debate. While some research advocates the use of constructed responses, others emphasize the potential of multiple-choice questions. Methodology: A sensitivity analysis was applied to extract useful knowledge from the relevance of the characteristics (i.e., the input variables) used for the data mining process to compute the score. Contribution: Such knowledge helps the teachers to decide which format they must consider with respect to the objectives and expected students results. Findings: The results shown a set of factors that influence the discrepancy between answers in both formats. Recommendations for Practitioners: Teachers can make an informed decision about whether to choose multiple-choice questions or constructed-response taking into account the results of this study. Recommendation for Researchers: In this study a block of exams with CR questions is verified to complement the area of learning, returning greater performance in the evaluation of students and improving the teaching-learning process. Impact on Society: The results of this research confirm the findings of several other researchers that the use of ICT and the application of MCQ is an added value in the evaluation process. In most cases the student is more likely to succeed with MCQ, however if the teacher prefers to evaluate with CR other research approaches are needed. Future Research: Future research must include other question formats.
Author Belo, Yolanda
Moro, Sérgio
Ramos, Pedro
Martins, António
Martinho Costa, Joana
Esmerado, Joaquim
Author_xml – sequence: 1
  givenname: Yolanda
  surname: Belo
  fullname: Belo, Yolanda
  organization: Instituto Universitário de Lisboa (ISCTE-IUL)
– sequence: 2
  givenname: Sérgio
  surname: Moro
  fullname: Moro, Sérgio
  organization: Instituto Universitário de Lisboa (ISCTE-IUL)
– sequence: 3
  givenname: António
  surname: Martins
  fullname: Martins, António
  organization: ISCTE - Instituto Universitário de Lisboa
– sequence: 4
  givenname: Pedro
  surname: Ramos
  fullname: Ramos, Pedro
  organization: ISCTE - Instituto Universitário de Lisboa
– sequence: 5
  givenname: Joana
  orcidid: 0000-0001-7400-0132
  surname: Martinho Costa
  fullname: Martinho Costa, Joana
  organization: Instituto Universitário de Lisboa (ISCTE-IUL)
– sequence: 6
  givenname: Joaquim
  surname: Esmerado
  fullname: Esmerado, Joaquim
  organization: ISCTE - Instituto Universitário de Lisboa
BookMark eNpdkF1LwzAUhoNMcM79h4B4JdWmTdP0Skb9xIkf7MK7kqYnW0bX1CR1-O9tnSjs3JyP9-E9h3OMRo1pAKEpCS8intHkktI0O0DjiLAkiAl7H_3VCTlCU-fWYR9pyjnJxmibm8Z520kPFX4D1_YtYGPxU1d73dYQ5CujJeDXDpzXvYpVr86cA-d0s8TXIGthhdefgF-sWVqx2Qzzx8Zsa6iWcIUXK-Gxdtiv_m1O0KEStYPpb56gxe3NIr8P5s93D_lsHsg4ZD4oUxGHwGOqqALKw76PiCBCqYiTWCYlLVVc0gqUyIBXjGdKCA5JSmUqMxZP0OnOtrXmY1hdrE1nm35jETHCGMtSFvfU-Y6S1jhnQRVSezGc6a3QdUHC4ue3xfDbnj7bo1urN8J-7XHf_Dx7yQ
CitedBy_id crossref_primary_10_1155_2021_4576750
ContentType Journal Article
Copyright 2019. This work is published under https://creativecommons.org/licenses/by-nc/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Copyright_xml – notice: 2019. This work is published under https://creativecommons.org/licenses/by-nc/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
DBID AAYXX
CITATION
0-V
3V.
7RQ
7XB
88B
8FK
8G5
ABUWG
AFKRA
ALSLI
AZQEC
BENPR
CCPQU
CJNVE
DWQXO
GNUQQ
GUQSH
M0P
M2O
MBDVC
PHGZM
PHGZT
PIMPY
PKEHL
PQEDU
PQEST
PQQKQ
PQUKI
PRINS
Q9U
U9A
DOI 10.28945/4479
DatabaseName CrossRef
ProQuest Social Sciences Premium Collection【Remote access available】
ProQuest Central (Corporate)
Career & Technical Education Database
ProQuest Central (purchase pre-March 2016)
Education Database (Alumni Edition)
ProQuest Central (Alumni) (purchase pre-March 2016)
Research Library (Alumni)
ProQuest Central (Alumni)
ProQuest Central
Social Science Premium Collection
ProQuest Central Essentials - QC
Proquest Central
ProQuest One
Education Collection
ProQuest Central Korea
ProQuest Central Student
ProQuest Research Library
Education Database (ProQuest)
ProQuest research library
Research Library (Corporate)
ProQuest Central Premium
ProQuest One Academic
ProQuest - Publicly Available Content Database
ProQuest One Academic Middle East (New)
ProQuest One Education
ProQuest One Academic Eastern Edition (DO NOT USE)
ProQuest One Academic (retired)
ProQuest One Academic UKI Edition
ProQuest Central China
ProQuest Central Basic
DatabaseTitle CrossRef
Publicly Available Content Database
ProQuest One Education
Research Library Prep
ProQuest Central Student
ProQuest One Academic Middle East (New)
ProQuest Central Essentials
ProQuest Central (Alumni Edition)
ProQuest One Community College
Research Library (Alumni Edition)
ProQuest Central China
ProQuest Central
ProQuest Central Korea
ProQuest Research Library
ProQuest Central (New)
Career and Technical Education (Alumni Edition)
Social Science Premium Collection
Education Collection
ProQuest Central Basic
ProQuest Education Journals
ProQuest One Academic Eastern Edition
ProQuest Career and Technical Education
ProQuest Social Sciences Premium Collection
ProQuest One Academic UKI Edition
ProQuest One Academic
ProQuest Education Journals (Alumni Edition)
ProQuest One Academic (New)
ProQuest Central (Alumni)
DatabaseTitleList Publicly Available Content Database
CrossRef
Database_xml – sequence: 1
  dbid: 7RQ
  name: Career & Technical Education Database (ProQuest)
  url: https://search.proquest.com/career
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Education
EISSN 2165-316X
EndPage 170
ExternalDocumentID 10_28945_4479
GroupedDBID 7RQ
8G5
AAHSB
AAKDD
AAYXX
ABUWG
AFFHD
AFKRA
ALMA_UNASSIGNED_HOLDINGS
ALSLI
AZQEC
BENPR
CCPQU
CITATION
CJNVE
DWQXO
EBS
EJD
GNUQQ
GUQSH
IPNFZ
KQ8
M0P
M2O
OK1
PHGZM
PHGZT
PIMPY
PQEDU
RIG
RNS
0-V
3V.
7XB
8FK
MBDVC
PKEHL
PQEST
PQQKQ
PQUKI
PRINS
Q9U
U9A
ID FETCH-LOGICAL-c306t-b7a30e834f4fe480b7a21a1aff2813c5b4bf3b4defa9e8d689faa8e574c7c963
IEDL.DBID 7RQ
ISICitedReferencesCount 1
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000566761900003&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 2165-3151
IngestDate Sat Nov 01 15:20:17 EDT 2025
Sat Nov 29 03:42:58 EST 2025
Tue Nov 18 22:31:06 EST 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed false
IsScholarly true
Language English
License https://creativecommons.org/licenses/by-nc/4.0
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c306t-b7a30e834f4fe480b7a21a1aff2813c5b4bf3b4defa9e8d689faa8e574c7c963
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0000-0001-7400-0132
OpenAccessLink https://www.proquest.com/docview/2616669763?pq-origsite=%requestingapplication%
PQID 2616669763
PQPubID 5324111
PageCount 18
ParticipantIDs proquest_journals_2616669763
crossref_citationtrail_10_28945_4479
crossref_primary_10_28945_4479
PublicationCentury 2000
PublicationDate 2019-00-00
20190101
PublicationDateYYYYMMDD 2019-01-01
PublicationDate_xml – year: 2019
  text: 2019-00-00
PublicationDecade 2010
PublicationPlace Santa Rosa
PublicationPlace_xml – name: Santa Rosa
PublicationTitle Journal of Information Technology Education. Innovations in Practice
PublicationYear 2019
Publisher Informing Science Institute
Publisher_xml – name: Informing Science Institute
SSID ssj0000778819
Score 2.064646
Snippet Aim/Purpose: This paper presents a data mining approach for analyzing responses to advanced declarative programming questions. The goal of this research is to...
Aim/Purpose This paper presents a data mining approach for analyzing responses to advanced declarative programming questions. The goal of this research is to...
SourceID proquest
crossref
SourceType Aggregation Database
Enrichment Source
Index Database
StartPage 153
SubjectTerms Educational Assessment
Educational Objectives
Feedback (Response)
High Achievement
Learning Analytics
Learning Processes
Measurement Techniques
Multiple choice
Outcomes of Education
Programming
Researchers
Taxonomy
Title Constructed Response or Multiple-Choice Questions for Assessing Declarative Programming Knowledge? That is the Question
URI https://www.proquest.com/docview/2616669763
Volume 18
WOSCitedRecordID wos000566761900003&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVPQU
  databaseName: Career & Technical Education Database (ProQuest)
  customDbUrl:
  eissn: 2165-316X
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0000778819
  issn: 2165-3151
  databaseCode: 7RQ
  dateStart: 20080101
  isFulltext: true
  titleUrlDefault: https://search.proquest.com/career
  providerName: ProQuest
– providerCode: PRVPQU
  databaseName: Education Database
  customDbUrl:
  eissn: 2165-316X
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0000778819
  issn: 2165-3151
  databaseCode: M0P
  dateStart: 20080101
  isFulltext: true
  titleUrlDefault: https://search.proquest.com/education
  providerName: ProQuest
– providerCode: PRVPQU
  databaseName: ProQuest Central
  customDbUrl:
  eissn: 2165-316X
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0000778819
  issn: 2165-3151
  databaseCode: BENPR
  dateStart: 20080101
  isFulltext: true
  titleUrlDefault: https://www.proquest.com/central
  providerName: ProQuest
– providerCode: PRVPQU
  databaseName: ProQuest Publicly Available Content Database
  customDbUrl:
  eissn: 2165-316X
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0000778819
  issn: 2165-3151
  databaseCode: PIMPY
  dateStart: 20080101
  isFulltext: true
  titleUrlDefault: http://search.proquest.com/publiccontent
  providerName: ProQuest
– providerCode: PRVPQU
  databaseName: Research Library
  customDbUrl:
  eissn: 2165-316X
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0000778819
  issn: 2165-3151
  databaseCode: M2O
  dateStart: 20080101
  isFulltext: true
  titleUrlDefault: https://search.proquest.com/pqrl
  providerName: ProQuest
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1bS8MwFA7qRATxLs7LyIOvxbVNm-RpeEXRzTqG6FNJ04QN5ja3oX_fc9psXh588aWQNJTCd3LunI-QkzCPpNKZ72UqhgAlsNYTfph7YWRiIwPN_YI64emet1ri-VkmLuE2cW2VM51YKOp8qDFHfgqePnjaYDzDxujNQ9YorK46Co1FUgFDHaOc8_bjPMdS5zgsHT3gwI-x4Bv5K2QNu5-FZNEpY9jA9d0c_dTGhYm53vjvz22Sdedc0rNSGrbIghlsIy-z6-HYIR_Iz1lMjDU5bZftsYYOx7Tp-gq9i-4QVAct8qAokRScWlpWhsHI0Uuj-25YOE3K1q5X3L-bpeYatNNVU9qbUPAs55_ZJZ3rq87FjeeoFzwNMcTUy7gK60aEzDJrmKjDOvCVr6wNAEodZSyzYcZyY5U0Io-FtEoJE3GmuYY7vUeWBsOB2SeU6wi51y2TAHuupFBRFqgY66maByyrkpMZAKl2Y8mRHaOfQnhS4JQiTlVSmx8blXM4fh84mkGTums4Sb9wOfj79SFZBU9IlrmVI7IESJhjsqzfp73JuFZIVY1Uzq9aSRtWzXqCz-AB9pLbZvLyCfKd3Yo
linkProvider ProQuest
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1NT9tAEB3RpCpIVSktqBRI9wBHK_F6be8eKlSFRonyoaiKKnqy1utdgUQTiKNG_Cj-IzP-CG0PvXHo0fZqZXufZ97OjOcBnAZZqLRJfS_VEW5QuHOe9IPMC0IbWcVN7BfSCd9H8WQiLy_VdAse6n9hqKyytomFoc4WhmLkbWT6yLTReQbnt3ceqUZRdrWW0ChhMbT3a9yy5Z8HF7i-Z5z3vs66fa9SFfAM0uOVl8Y66FgZCCecFbKDx9zXvnaO412aMBWpC1KRWaeVlVkkldNa2jAWJjYIV5z2BTQFYb0BzelgPP2xCep0YurOTpSb-xFlmEP_FbymcmupRNgWgirGfvd_f5r_wqf1dv-zt_EW3lTkmX0p0b4HW3b-jnSnqxqV97Am_dGiI67N2Ley_NeyxZKNq7pJr3u1QNPIijgvfXEMSTsrM9_oxNmFNTdVM3Q2LUvXftL5YR16PGezK71i1zlD5ryZZh9mz_HYB9CYL-b2A7DYhKQt74RCWGdaSR2mXEeULzYxF-khnNbrnZiq7Tqpf9wkuP0qYJEQLA6htRl2W_YZ-XvAcY2EpDIzefIEg4__vvwJtvuz8SgZDSbDI9hB1qfKONIxNHBV7Am8NL9W1_myVUGaQfLMsHkERNY4yA
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Constructed+Response+or+Multiple-Choice+Questions+for+Assessing+Declarative+Programming+Knowledge%3F+That+is+the+Question&rft.jtitle=Journal+of+Information+Technology+Education.+Innovations+in+Practice&rft.au=Belo%2C+Yolanda&rft.au=Moro%2C+S%C3%A9rgio&rft.au=Martins%2C+Ant%C3%B3nio&rft.au=Ramos%2C+Pedro&rft.date=2019&rft.issn=2165-3151&rft.eissn=2165-316X&rft.volume=18&rft.spage=153&rft.epage=170&rft_id=info:doi/10.28945%2F4479&rft.externalDBID=n%2Fa&rft.externalDocID=10_28945_4479
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2165-3151&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2165-3151&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2165-3151&client=summon