Disengaged response behavior when the response button is blocked: Evaluation of a micro-intervention

In large-scale assessments, disengaged participants might rapidly guess on items or skip items, which can affect the score interpretation’s validity. This study analyzes data from a linear computer-based assessment to evaluate a micro-intervention that blocked the possibility to respond for 2 s. The...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Frontiers in psychology Ročník 13; s. 954532
Hlavní autoři: Persic-Beck, Lothar, Goldhammer, Frank, Kroehne, Ulf
Médium: Journal Article
Jazyk:angličtina
Vydáno: Switzerland Frontiers Media S.A 03.11.2022
Témata:
log data analysis
process data analysis
Psychology
rapid guessing behavior
rapid omission
rapid responding
test taking behavior
rapid responding
rapid guessing behavior
finite state machine
log data analysis
rapid omission
test taking behavior
process data analysis
ISSN:1664-1078, 1664-1078
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:In large-scale assessments, disengaged participants might rapidly guess on items or skip items, which can affect the score interpretation’s validity. This study analyzes data from a linear computer-based assessment to evaluate a micro-intervention that blocked the possibility to respond for 2 s. The blocked response was implemented to prevent participants from accidental navigation and as a naive attempt to prevent rapid guesses and rapid omissions. The response process was analyzed by interpreting log event sequences within a finite-state machine approach. Responses were assigned to different response classes based on the event sequence. Additionally, post hoc methods for detecting rapid responses based on response time thresholds were applied to validate the classification. Rapid guesses and rapid omissions could be distinguished from accidental clicks by the log events following the micro-intervention. Results showed that the blocked response interfered with rapid responses but hardly led to behavioral changes. However, the blocked response could improve the post hoc detection of rapid responding by identifying responses that narrowly exceed time-bound thresholds. In an assessment context, it is desirable to prevent participants from accidentally skipping items, which in itself may lead to an increasing popularity of initially blocking responses. If, however, data from those assessments is analyzed for rapid responses, additional log data information should be considered.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
This article was submitted to Quantitative Psychology and Measurement, a section of the journal Frontiers in Psychology
Reviewed by: Hong Jiao, University of Maryland, College Park, United States; Michalis P. Michaelides, University of Cyprus, Cyprus
Edited by: Erin M. Buchanan, Harrisburg University of Science and Technology, United States
ISSN:1664-1078
1664-1078
DOI:10.3389/fpsyg.2022.954532