Reaction time effects in lab- versus Web-based research: Experimental evidence

Although Web-based research is now commonplace, it continues to spur skepticism from reviewers and editors, especially whenever reaction times are of primary interest. Such persistent preconceptions are based on arguments referring to increased variation, the limits of certain software and technolog...

Full description

Saved in:
Bibliographic Details
Published in:Behavior research methods Vol. 48; no. 4; pp. 1718 - 1724
Main Author: Hilbig, Benjamin E.
Format: Journal Article
Language:English
Published: New York Springer US 01.12.2016
Subjects:
Behavioral Research - methods
Behavioral Science and Psychology
Cognitive Psychology
Data Collection
Humans
Internet
Psychology
Reaction Time - physiology
Software
Young Adult
Reaction time
Response latency
Internet
Web
Word frequency effect
ISSN:1554-3528, 1554-3528
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Although Web-based research is now commonplace, it continues to spur skepticism from reviewers and editors, especially whenever reaction times are of primary interest. Such persistent preconceptions are based on arguments referring to increased variation, the limits of certain software and technologies, and a noteworthy lack of comparisons (between Web and lab) in fully randomized experiments. To provide a critical test, participants were randomly assigned to complete a lexical decision task either (a) in the lab using standard experimental software (E-Prime), (b) in the lab using a browser-based version (written in HTML and JavaScript), or (c) via the Web using the same browser-based version. The classical word frequency effect was typical in size and corresponded to a very large effect in all three conditions. There was no indication that the Web- or browser-based data collection was in any way inferior. In fact, if anything, a larger effect was obtained in the browser-based conditions than in the condition relying on standard experimental software. No differences between Web and lab (within the browser-based conditions) could be observed, thus disconfirming any substantial influence of increased technical or situational variation. In summary, the present experiment contradicts the still common preconception that reaction time effects of only a few hundred milliseconds cannot be detected in Web experiments.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:1554-3528
1554-3528
DOI:10.3758/s13428-015-0678-9