Reaction time effects in lab- versus Web-based research: Experimental evidence

Although Web-based research is now commonplace, it continues to spur skepticism from reviewers and editors, especially whenever reaction times are of primary interest. Such persistent preconceptions are based on arguments referring to increased variation, the limits of certain software and technolog...

Full description

Saved in:
Bibliographic Details
Published in:Behavior research methods Vol. 48; no. 4; pp. 1718 - 1724
Main Author: Hilbig, Benjamin E.
Format: Journal Article
Language:English
Published: New York Springer US 01.12.2016
Subjects:
Behavioral Research - methods
Behavioral Science and Psychology
Cognitive Psychology
Data Collection
Humans
Internet
Psychology
Reaction Time - physiology
Software
Young Adult
Reaction time
Response latency
Internet
Web
Word frequency effect
ISSN:1554-3528, 1554-3528
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Although Web-based research is now commonplace, it continues to spur skepticism from reviewers and editors, especially whenever reaction times are of primary interest. Such persistent preconceptions are based on arguments referring to increased variation, the limits of certain software and technologies, and a noteworthy lack of comparisons (between Web and lab) in fully randomized experiments. To provide a critical test, participants were randomly assigned to complete a lexical decision task either (a) in the lab using standard experimental software (E-Prime), (b) in the lab using a browser-based version (written in HTML and JavaScript), or (c) via the Web using the same browser-based version. The classical word frequency effect was typical in size and corresponded to a very large effect in all three conditions. There was no indication that the Web- or browser-based data collection was in any way inferior. In fact, if anything, a larger effect was obtained in the browser-based conditions than in the condition relying on standard experimental software. No differences between Web and lab (within the browser-based conditions) could be observed, thus disconfirming any substantial influence of increased technical or situational variation. In summary, the present experiment contradicts the still common preconception that reaction time effects of only a few hundred milliseconds cannot be detected in Web experiments.
AbstractList Although Web-based research is now commonplace, it continues to spur skepticism from reviewers and editors, especially whenever reaction times are of primary interest. Such persistent preconceptions are based on arguments referring to increased variation, the limits of certain software and technologies, and a noteworthy lack of comparisons (between Web and lab) in fully randomized experiments. To provide a critical test, participants were randomly assigned to complete a lexical decision task either (a) in the lab using standard experimental software (E-Prime), (b) in the lab using a browser-based version (written in HTML and JavaScript), or (c) via the Web using the same browser-based version. The classical word frequency effect was typical in size and corresponded to a very large effect in all three conditions. There was no indication that the Web- or browser-based data collection was in any way inferior. In fact, if anything, a larger effect was obtained in the browser-based conditions than in the condition relying on standard experimental software. No differences between Web and lab (within the browser-based conditions) could be observed, thus disconfirming any substantial influence of increased technical or situational variation. In summary, the present experiment contradicts the still common preconception that reaction time effects of only a few hundred milliseconds cannot be detected in Web experiments.
Although Web-based research is now commonplace, it continues to spur skepticism from reviewers and editors, especially whenever reaction times are of primary interest. Such persistent preconceptions are based on arguments referring to increased variation, the limits of certain software and technologies, and a noteworthy lack of comparisons (between Web and lab) in fully randomized experiments. To provide a critical test, participants were randomly assigned to complete a lexical decision task either (a) in the lab using standard experimental software (E-Prime), (b) in the lab using a browser-based version (written in HTML and JavaScript), or (c) via the Web using the same browser-based version. The classical word frequency effect was typical in size and corresponded to a very large effect in all three conditions. There was no indication that the Web- or browser-based data collection was in any way inferior. In fact, if anything, a larger effect was obtained in the browser-based conditions than in the condition relying on standard experimental software. No differences between Web and lab (within the browser-based conditions) could be observed, thus disconfirming any substantial influence of increased technical or situational variation. In summary, the present experiment contradicts the still common preconception that reaction time effects of only a few hundred milliseconds cannot be detected in Web experiments.Although Web-based research is now commonplace, it continues to spur skepticism from reviewers and editors, especially whenever reaction times are of primary interest. Such persistent preconceptions are based on arguments referring to increased variation, the limits of certain software and technologies, and a noteworthy lack of comparisons (between Web and lab) in fully randomized experiments. To provide a critical test, participants were randomly assigned to complete a lexical decision task either (a) in the lab using standard experimental software (E-Prime), (b) in the lab using a browser-based version (written in HTML and JavaScript), or (c) via the Web using the same browser-based version. The classical word frequency effect was typical in size and corresponded to a very large effect in all three conditions. There was no indication that the Web- or browser-based data collection was in any way inferior. In fact, if anything, a larger effect was obtained in the browser-based conditions than in the condition relying on standard experimental software. No differences between Web and lab (within the browser-based conditions) could be observed, thus disconfirming any substantial influence of increased technical or situational variation. In summary, the present experiment contradicts the still common preconception that reaction time effects of only a few hundred milliseconds cannot be detected in Web experiments.
Author Hilbig, Benjamin E.
Author_xml – sequence: 1
  givenname: Benjamin E.
  surname: Hilbig
  fullname: Hilbig, Benjamin E.
  email: hilbig@uni-landau.de
  organization: Cognitive Psychology Laboratory, Department of Psychology, University of Koblenz-Landau, Max Planck Institute for Research on Collective Goods
BackLink https://www.ncbi.nlm.nih.gov/pubmed/26542972$$D View this record in MEDLINE/PubMed
BookMark eNp9kEtLxDAUhYOM6Pj4AW4kSzfRNGna1J0M4wNEQRSXIY9brXTSMWlF_70pVRAXs8qFfOdw-PbQzHceEDrK6CkvhTyLGc-ZJDQThBalJNUWmmdC5IQLJmd_7l20F-MbpVyyLN9Bu6wQOatKNkd3D6Bt33Qe980KMNQ12D7ixuNWG4I_IMQh4mcwxOgIDgeIoIN9PcfLzzWElPG9bjF8NA68hQO0Xes2wuHPu4-eLpePi2tye391s7i4JVYw2hNpOEuDa6lrRrU1rtTSpEFVLSSn6c_KqnaU547mZmQAJDBXSgOlpM7xfXQy9a5D9z5A7NWqiRbaVnvohqgyyYqCM1FWCT3-QQezAqfWabQOX-rXQQLKCbChizFArWzT69FJH3TTqoyq0baabKtkW4221Vid_Uv-lm_KsCkTE-tfIKi3bgg-ydoQ-ga_55D-
CitedBy_id crossref_primary_10_1016_j_jrp_2022_104234
crossref_primary_10_1038_s41562_022_01327_5
crossref_primary_10_1007_s12369_022_00926_6
crossref_primary_10_1016_j_addbeh_2021_107123
crossref_primary_10_1016_j_rmal_2025_100228
crossref_primary_10_1515_phon_2022_2022
crossref_primary_10_1007_s11257_023_09383_w
crossref_primary_10_3389_fpsyg_2024_1440057
crossref_primary_10_3758_s13423_018_1555_1
crossref_primary_10_3758_s13428_016_0783_4
crossref_primary_10_1080_13803395_2021_1982867
crossref_primary_10_1016_j_tics_2017_06_007
crossref_primary_10_1177_17470218221103500
crossref_primary_10_1017_S136672892510028X
crossref_primary_10_3758_s13428_023_02155_9
crossref_primary_10_1080_0361073X_2022_2137358
crossref_primary_10_1097_PSY_0000000000000534
crossref_primary_10_1016_j_cognition_2020_104530
crossref_primary_10_1162_opmi_a_00107
crossref_primary_10_1007_s10608_020_10180_7
crossref_primary_10_7717_peerj_11266
crossref_primary_10_3758_s13428_019_01283_5
crossref_primary_10_1016_j_physbeh_2021_113639
crossref_primary_10_3389_fnbeh_2022_946263
crossref_primary_10_1038_s41598_022_10986_3
crossref_primary_10_1016_j_rmal_2025_100242
crossref_primary_10_1038_s41598_023_41900_0
crossref_primary_10_1177_17470218231154884
crossref_primary_10_3758_s13428_018_1155_z
crossref_primary_10_3389_fpsyg_2021_734877
crossref_primary_10_1016_j_humov_2021_102786
crossref_primary_10_1007_s11251_022_09602_8
crossref_primary_10_1016_j_cogpsych_2017_08_003
crossref_primary_10_3390_ijerph19042201
crossref_primary_10_1111_sjop_12494
crossref_primary_10_3758_s13428_018_1036_5
crossref_primary_10_3758_s13428_022_01885_6
crossref_primary_10_3758_s13421_023_01450_2
crossref_primary_10_3389_fpsyg_2024_1382668
crossref_primary_10_3390_fi13100245
crossref_primary_10_1145_3733053
crossref_primary_10_1080_14992027_2024_2402825
crossref_primary_10_3758_s13428_023_02131_3
crossref_primary_10_3758_s13428_024_02533_x
crossref_primary_10_1080_20445911_2022_2089153
crossref_primary_10_3389_fpsyg_2021_675641
crossref_primary_10_1177_21677026241258380
crossref_primary_10_1016_j_actpsy_2025_104726
crossref_primary_10_3390_bs11100128
crossref_primary_10_1177_10738584211017018
crossref_primary_10_1080_00223891_2019_1663357
crossref_primary_10_3390_brainsci10040251
crossref_primary_10_1016_j_actpsy_2022_103666
crossref_primary_10_1016_j_actpsy_2020_103043
crossref_primary_10_3389_fpsyg_2020_01839
crossref_primary_10_3389_fpsyg_2024_1359793
crossref_primary_10_1007_s10683_017_9528_1
crossref_primary_10_1371_journal_pone_0303384
crossref_primary_10_1109_ACCESS_2021_3066383
crossref_primary_10_3758_s13428_024_02454_9
crossref_primary_10_3758_s13428_022_01989_z
crossref_primary_10_1007_s41237_023_00193_3
crossref_primary_10_1080_13218719_2020_1793819
crossref_primary_10_1016_j_psychsport_2025_102913
crossref_primary_10_1371_journal_pone_0273913
crossref_primary_10_2196_65836
crossref_primary_10_1016_j_cognition_2023_105606
crossref_primary_10_1038_s41598_023_35381_4
crossref_primary_10_3389_fnagi_2022_804937
crossref_primary_10_3389_fpsyg_2019_02883
crossref_primary_10_3390_psych2030012
crossref_primary_10_5265_jcogpsy_2415
crossref_primary_10_1002_eat_23280
crossref_primary_10_1177_17470218231203951
crossref_primary_10_1017_S0272263124000214
crossref_primary_10_3758_s13428_022_02057_2
crossref_primary_10_1177_1029864920911127
crossref_primary_10_3389_fpsyg_2017_01582
crossref_primary_10_1080_25742442_2022_2094657
crossref_primary_10_1177_21582440231173823
crossref_primary_10_1016_j_psychsport_2025_102808
crossref_primary_10_1016_j_actpsy_2023_103944
crossref_primary_10_1080_13506285_2023_2188336
crossref_primary_10_1177_0018720818787126
crossref_primary_10_3389_fpsyg_2021_800543
crossref_primary_10_1007_s10608_023_10399_0
crossref_primary_10_2139_ssrn_2648686
crossref_primary_10_3389_fpsyg_2025_1610265
crossref_primary_10_3390_medicina60111890
crossref_primary_10_3758_s13428_016_0758_5
crossref_primary_10_1177_13621688211039023
crossref_primary_10_1007_s10508_023_02676_6
crossref_primary_10_1177_1747021819840583
crossref_primary_10_1027_1618_3169_a000599
crossref_primary_10_1098_rsos_210550
crossref_primary_10_1177_1046878118805515
crossref_primary_10_1371_journal_pone_0264034
crossref_primary_10_1016_j_rmal_2023_100045
crossref_primary_10_3389_fspor_2021_685286
crossref_primary_10_1121_10_0020064
crossref_primary_10_1016_j_rmal_2025_100247
crossref_primary_10_1177_17470218221096148
crossref_primary_10_3758_s13428_021_01686_3
crossref_primary_10_1038_s41598_025_95717_0
crossref_primary_10_1371_journal_pone_0259367
crossref_primary_10_3389_fpsyg_2022_880375
crossref_primary_10_1002_acp_3759
crossref_primary_10_1093_geronb_gbab143
crossref_primary_10_1186_s40359_021_00712_5
crossref_primary_10_3758_s13428_024_02557_3
crossref_primary_10_1007_s00426_022_01650_6
crossref_primary_10_1007_s11469_018_9991_8
crossref_primary_10_3758_s13414_023_02652_1
crossref_primary_10_3758_s13414_023_02755_9
crossref_primary_10_1016_j_cortex_2024_11_019
crossref_primary_10_3390_clockssleep4030033
crossref_primary_10_3389_fpsyg_2021_710663
crossref_primary_10_3758_s13428_021_01573_x
crossref_primary_10_1016_j_ijhm_2021_103083
Cites_doi 10.1016/S0022-5371(83)80004-8
10.1016/j.chb.2004.09.010
10.3758/BF03196267
10.1037/0003-066X.59.2.93
10.3758/s13428-011-0066-z
10.1037/12076-013
10.3758/s13428-014-0458-y
10.3758/BF03202553
10.1037/0033-2909.86.3.446
10.1016/S0022-5371(70)80091-3
10.3758/BRM.41.3.598
10.3758/BF03195440
10.3758/s13423-012-0296-9
10.3758/BRM.41.4.1183
10.3758/BF03193004
10.1201/b10855-37
10.1037/0003-066X.59.2.105
10.3758/s13428-014-0471-1
10.1016/j.jrp.2005.01.006
10.1111/1467-9280.00296
10.3758/BF03195364
10.3758/BRM.41.1.12
10.1146/annurev.psych.57.102904.190048
10.3758/BF03193146
10.3758/s13428-011-0069-9
10.1177/0894439311415604
10.3758/BF03200724
10.3758/s13428-013-0345-y
10.1146/annurev.psych.55.090902.141601
10.1371/journal.pone.0057410
10.1037/0003-066x.59.2.105
10.3758/bf03193004
10.3758/bf03196267
10.3758/s13428-015-0567-2
10.3758/bf03202553
10.1177/08939302020003002
10.1037/a0019999
10.3758/bf03200724
ContentType Journal Article
Copyright Psychonomic Society, Inc. 2015
Copyright_xml – notice: Psychonomic Society, Inc. 2015
DBID AAYXX
CITATION
CGR
CUY
CVF
ECM
EIF
NPM
7X8
DOI 10.3758/s13428-015-0678-9
DatabaseName CrossRef
Medline
MEDLINE
MEDLINE (Ovid)
MEDLINE
MEDLINE
PubMed
MEDLINE - Academic
DatabaseTitle CrossRef
MEDLINE
Medline Complete
MEDLINE with Full Text
PubMed
MEDLINE (Ovid)
MEDLINE - Academic
DatabaseTitleList
MEDLINE
MEDLINE - Academic
Database_xml – sequence: 1
  dbid: NPM
  name: PubMed
  url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed
  sourceTypes: Index Database
– sequence: 2
  dbid: 7X8
  name: MEDLINE - Academic
  url: https://search.proquest.com/medline
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Psychology
EISSN 1554-3528
EndPage 1724
ExternalDocumentID 26542972
10_3758_s13428_015_0678_9
Genre Journal Article
GroupedDBID ---
-55
-5G
-BR
-DZ
-EM
-ET
-~C
-~X
0-V
06D
0R~
0VY
199
1N0
203
23N
2J2
2JN
2JY
2KG
2KM
2LR
2VQ
30V
3V.
4.4
406
408
40E
53G
5GY
7X7
875
88E
8AO
8FI
8FJ
8G5
8TC
8UJ
95.
96X
AAAVM
AABHQ
AACDK
AAHNG
AAIAL
AAJBT
AAJKR
AAKPC
AANZL
AARTL
AASML
AATNV
AATVU
AAUYE
AAWCG
AAYIU
AAYQN
AAYTO
AAYZH
AAZMS
ABAKF
ABDZT
ABECU
ABFTV
ABHLI
ABIVO
ABJNI
ABJOX
ABJUD
ABKCH
ABMQK
ABNWP
ABPLI
ABPPZ
ABQBU
ABQSL
ABSXP
ABTEG
ABTHY
ABTKH
ABTMW
ABUWG
ABXPI
ACAOD
ACBXY
ACDTI
ACGFS
ACHQT
ACHSB
ACHXU
ACIWK
ACKIV
ACKNC
ACMDZ
ACMLO
ACNCT
ACOKC
ACPIV
ACPRK
ACZOJ
ADBBV
ADHHG
ADHIR
ADINQ
ADKNI
ADKPE
ADURQ
ADYFF
ADZKW
AEBTG
AEFQL
AEGAL
AEGNC
AEJHL
AEJRE
AEKMD
AEMSY
AEOHA
AEPYU
AESKC
AETCA
AEVLU
AEXYK
AFBBN
AFFNX
AFKRA
AFQWF
AFRAH
AFWTZ
AFZKB
AGAYW
AGDGC
AGJBK
AGMZJ
AGQEE
AGQMX
AGRTI
AGWIL
AGWZB
AGYKE
AHAVH
AHBYD
AHMBA
AHSBF
AHYZX
AIAKS
AIGIU
AIIXL
AILAN
AITGF
AJBLW
AJRNO
AJZVZ
ALIPV
ALMA_UNASSIGNED_HOLDINGS
ALSLI
AMKLP
AMXSW
AMYLF
AMYQR
AOCGG
ARALO
ARMRJ
ASPBG
AVWKF
AXYYD
AYQZM
AZFZN
AZQEC
B-.
BAWUL
BENPR
BGNMA
BPHCQ
BVXVI
C1A
CAG
CCPQU
COF
CSCUP
DDRTE
DIK
DNIVK
DPUIP
DWQXO
E3Z
EBD
EBLON
EBS
EIOEI
EJD
EMOBN
ESBYG
F5P
FEDTE
FERAY
FFXSO
FIGPU
FINBP
FNLPD
FRRFC
FSGXE
FYUFA
GGCAI
GGRSB
GJIRD
GNUQQ
GNWQR
GQ3
GQ6
GQ7
GUQSH
H13
HF~
HMCUK
HMJXF
HRMNR
HVGLF
HZ~
H~9
IAO
IHR
IKXTQ
INH
IPY
IRVIT
ITC
ITM
IWAJR
J-C
JBSCW
JZLTJ
KOV
LLZTM
M1P
M2M
M2O
M2R
M4Y
MVM
N2Q
N9A
NB0
NPVJJ
NQJWS
NU0
O9-
O93
O9G
O9J
OHT
OK1
P2P
P9L
PADUT
PF-
PQQKQ
PROAC
PSQYO
PSYQQ
PT4
R9I
RIG
ROL
RPV
RSV
S16
S1Z
S27
S3B
SBS
SBU
SCLPG
SHX
SISQX
SJYHP
SNE
SNPRN
SNX
SOHCF
SOJ
SPISZ
SRMVM
SSLCW
SSXJD
STPWE
SV3
SZN
T13
TN5
TR2
TSG
TUC
TUS
U2A
U9L
UG4
UKHRP
UOJIU
UPT
UTJUX
UZXMN
VFIZW
VXZ
W48
WH7
WK8
XJT
XOL
XSW
Z7R
Z7S
Z7W
Z81
Z83
Z88
Z8N
Z92
ZMTXR
ZOVNA
ZUP
AAPKM
AAYXX
ABBRH
ABDBE
ABFSG
ABRTQ
ACSTC
ADHKG
AEZWR
AFDZB
AFFHD
AFHIU
AFOHR
AGQPQ
AHPBZ
AHWEU
AIXLP
ATHPR
AYFIA
CITATION
PHGZM
PHGZT
PJZUB
PPXIY
PRQQA
CGR
CUY
CVF
ECM
EIF
NPM
7X8
ID FETCH-LOGICAL-c520t-8b32678f8af20acbd7a8b2979f5830326c89fd034d04baf20ee8e2d78be780dd3
IEDL.DBID RSV
ISICitedReferencesCount 139
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000392719000040&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 1554-3528
IngestDate Fri Sep 05 10:57:38 EDT 2025
Wed Feb 19 02:40:52 EST 2025
Sat Nov 29 02:18:05 EST 2025
Tue Nov 18 20:44:18 EST 2025
Fri Feb 21 02:37:00 EST 2025
IsDoiOpenAccess false
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue 4
Keywords Reaction time
Response latency
Internet
Web
Word frequency effect
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c520t-8b32678f8af20acbd7a8b2979f5830326c89fd034d04baf20ee8e2d78be780dd3
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
OpenAccessLink https://link.springer.com/content/pdf/10.3758/s13428-015-0678-9.pdf
PMID 26542972
PQID 1826632579
PQPubID 23479
PageCount 7
ParticipantIDs proquest_miscellaneous_1826632579
pubmed_primary_26542972
crossref_citationtrail_10_3758_s13428_015_0678_9
crossref_primary_10_3758_s13428_015_0678_9
springer_journals_10_3758_s13428_015_0678_9
PublicationCentury 2000
PublicationDate 2016-12-01
PublicationDateYYYYMMDD 2016-12-01
PublicationDate_xml – month: 12
  year: 2016
  text: 2016-12-01
  day: 01
PublicationDecade 2010
PublicationPlace New York
PublicationPlace_xml – name: New York
– name: United States
PublicationTitle Behavior research methods
PublicationTitleAbbrev Behav Res
PublicationTitleAlternate Behav Res Methods
PublicationYear 2016
Publisher Springer US
Publisher_xml – name: Springer US
References Gosling, Vazire, Srivastava, John (CR15) 2004; 59
Reips (CR30) 2002; 20
de Leeuw (CR9) 2015; 47
Linnman, Carlbring, Ahman, Andersson, Andersson (CR21) 2006; 22
Simcox, Fiez (CR36) 2014; 46
Crump, McDonnell, Gureckis (CR8) 2013; 8
Reips, Birnbaum, Proctor, Vu (CR31) 2011
Birnbaum (CR2) 2004; 55
Plant, Turner (CR26) 2009; 41
Eichstaedt (CR11) 2001; 33
CR32
Ollesch, Heineken, Schulte (CR24) 2006; 1
Rubenstein, Garfield, Millikan (CR34) 1970; 9
Cronk, West (CR7) 2002; 34
Chuah, Drasgow, Roberts (CR4) 2006; 40
Gordon (CR14) 1983; 22
Ihme, Lemke, Lieder, Martin, Muller, Schmidt (CR17) 2009; 41
Brand, Bradley (CR3) 2012; 30
Skitka, Sargis (CR37) 2006; 57
Kraut, Olson, Banaji, Bruckman, Cohen, Couper (CR19) 2004; 59
Keller, Gunasekharan, Mayo, Corley (CR18) 2009; 41
Reimers, Stewart (CR28) 2007; 39
Hecht, Oesker, Kaiser, Civelek, Stecker (CR16) 1999; 31
Lang, John, Lütdke, Schupp, Wagner (CR20) 2011; 43
de Leeuw, Motz (CR10) 2015
Plant, Hammond, Whitehouse (CR25) 2003; 35
McGraw, Tew, Williams (CR22) 2000; 11
Albrecht, Vorberg (CR1) 2010; 36
Corley, Scheepers (CR6) 2002; 9
Neath, Earle, Hallett, Surprenant (CR23) 2011; 43
Cohen (CR5) 1988
Germine, Nakayama, Duchaine, Chabris, Chatterjee, Wilmer (CR13) 2012; 19
Schneider, Eschman, Zuccolotto (CR35) 2002
Ratcliff (CR27) 1979; 86
Reimers, Stewart (CR29) 2015; 47
Faul, Erdfelder, Lang, Buchner (CR12) 2007; 39
Reips, Krantz, Gosling, Johnson (CR33) 2010
J Cohen (678_CR5) 1988
H Hecht (678_CR16) 1999; 31
RR Plant (678_CR26) 2009; 41
U-D Reips (678_CR31) 2011
JM Ihme (678_CR17) 2009; 41
L Germine (678_CR13) 2012; 19
J Eichstaedt (678_CR11) 2001; 33
MH Birnbaum (678_CR2) 2004; 55
S Reimers (678_CR29) 2015; 47
S Reimers (678_CR28) 2007; 39
JR Leeuw de (678_CR10) 2015
A Brand (678_CR3) 2012; 30
KO McGraw (678_CR22) 2000; 11
BC Cronk (678_CR7) 2002; 34
I Neath (678_CR23) 2011; 43
M Corley (678_CR6) 2002; 9
MJC Crump (678_CR8) 2013; 8
H Ollesch (678_CR24) 2006; 1
T Simcox (678_CR36) 2014; 46
678_CR32
F Faul (678_CR12) 2007; 39
JR Leeuw de (678_CR9) 2015; 47
U-D Reips (678_CR30) 2002; 20
B Gordon (678_CR14) 1983; 22
C Linnman (678_CR21) 2006; 22
F Keller (678_CR18) 2009; 41
U-D Reips (678_CR33) 2010
LJ Skitka (678_CR37) 2006; 57
T Albrecht (678_CR1) 2010; 36
H Rubenstein (678_CR34) 1970; 9
SC Chuah (678_CR4) 2006; 40
FR Lang (678_CR20) 2011; 43
R Kraut (678_CR19) 2004; 59
R Ratcliff (678_CR27) 1979; 86
W Schneider (678_CR35) 2002
SD Gosling (678_CR15) 2004; 59
RR Plant (678_CR25) 2003; 35
References_xml – volume: 22
  start-page: 24
  year: 1983
  end-page: 44
  ident: CR14
  article-title: Lexical access and lexical decision: Mechanisms of frequency sensitivity
  publication-title: Journal of Verbal Learning and Verbal Behavior
  doi: 10.1016/S0022-5371(83)80004-8
– volume: 22
  start-page: 448
  year: 2006
  end-page: 455
  ident: CR21
  article-title: The Stroop effect on the internet
  publication-title: Computers in Human Behavior
  doi: 10.1016/j.chb.2004.09.010
– volume: 20
  start-page: 241
  year: 2002
  end-page: 249
  ident: CR30
  article-title: Internet-based psychological experimenting: Five dos and five don’t
  publication-title: Social Science Computer Review
– volume: 9
  start-page: 126
  year: 2002
  end-page: 131
  ident: CR6
  article-title: Syntactic priming in English sentence production: Categorical and latency evidence from an Internet-based study
  publication-title: Psychonomic Bulletin & Review
  doi: 10.3758/BF03196267
– volume: 59
  start-page: 93
  year: 2004
  end-page: 104
  ident: CR15
  article-title: Should we trust web-based studies? A comparative analysis of six preconceptions about internet questionnaires
  publication-title: American Psychologist
  doi: 10.1037/0003-066X.59.2.93
– volume: 36
  start-page: 1339
  year: 2010
  end-page: 1345
  ident: CR1
  article-title: Long-lasting effects of briefly flashed words and pseudowords in ultrarapid serial visual presentation
  publication-title: Journal of Experimental Psychology: Learning, Memory, and Cognition
– volume: 43
  start-page: 548
  year: 2011
  end-page: 567
  ident: CR20
  article-title: Short assessment of the Big Five: Robust across survey methods except telephone interviewing
  publication-title: Behavior Research Methods
  doi: 10.3758/s13428-011-0066-z
– start-page: 193
  year: 2010
  end-page: 216
  ident: CR33
  article-title: Conducting true experiments on the Web
  publication-title: Advanced methods for conducting online behavioral research
  doi: 10.1037/12076-013
– volume: 47
  start-page: 1
  year: 2015
  end-page: 12
  ident: CR9
  article-title: jsPsych: A JavaScript library for creating behavioral experiments in a Web browser
  publication-title: Behavior Research Methods
  doi: 10.3758/s13428-014-0458-y
– volume: 8
  start-page: 1
  issue: e57410
  year: 2013
  end-page: 18
  ident: CR8
  article-title: Evaluating Amazon’s Mechanical Turk as a Tool for Experimental Behavioral Research
  publication-title: PLoS ONE
– volume: 35
  start-page: 276
  year: 2003
  end-page: 284
  ident: CR25
  article-title: How choice of mouse may affect response timing in psychological studies
  publication-title: Behavior Research Methods, Instruments, & Computers
  doi: 10.3758/BF03202553
– volume: 86
  start-page: 446
  year: 1979
  end-page: 461
  ident: CR27
  article-title: Group reaction time distributions and an analysis of distribution statistics
  publication-title: Psychological Bulletin
  doi: 10.1037/0033-2909.86.3.446
– volume: 9
  start-page: 487
  year: 1970
  end-page: 494
  ident: CR34
  article-title: Homographic entries in the internal lexicon
  publication-title: Journal of Verbal Learning and Verbal Behavior
  doi: 10.1016/S0022-5371(70)80091-3
– volume: 41
  start-page: 598
  year: 2009
  end-page: 614
  ident: CR26
  article-title: Millisecond precision psychological research in a world of commodity computers: New hardware, new problems?
  publication-title: Behavior Research Methods
  doi: 10.3758/BRM.41.3.598
– volume: 34
  start-page: 177
  year: 2002
  end-page: 180
  ident: CR7
  article-title: Personality research on the Internet: A comparison of Web-based and traditional instruments in take-home and in-class settings
  publication-title: Behavior Research Methods, Instruments, & Computers
  doi: 10.3758/BF03195440
– volume: 19
  start-page: 847
  year: 2012
  end-page: 857
  ident: CR13
  article-title: Is the Web as good as the lab? Comparable performance from Web and lab in cognitive/perceptual experiments
  publication-title: Psychonomic Bulletin & Review
  doi: 10.3758/s13423-012-0296-9
– volume: 41
  start-page: 1183
  year: 2009
  end-page: 1189
  ident: CR17
  article-title: Comparison of ability tests administered online and in the laboratory
  publication-title: Behavior Research Methods
  doi: 10.3758/BRM.41.4.1183
– year: 2002
  ident: CR35
  publication-title: E-Prime: User’s guide
– year: 2015
  ident: CR10
  article-title: Psychophysics in a Web browser? Comparing response times collected with JavaScript and Psychophysics Toolbox in a visual search task
  publication-title: Behavior Research Methods
– volume: 1
  start-page: 71
  year: 2006
  end-page: 81
  ident: CR24
  article-title: Physical or virtual presence of the experimenter: Psychological online-experiments in different settings
  publication-title: International Journal of Internet Science
– volume: 39
  start-page: 365
  year: 2007
  end-page: 370
  ident: CR28
  article-title: Adobe flash as a medium for online experimentation: A test of reaction time measurement capabilities
  publication-title: Behavior Research Methods
  doi: 10.3758/BF03193004
– start-page: 563
  year: 2011
  end-page: 585
  ident: CR31
  article-title: Behavioral research and data collection via the internet
  publication-title: The handbook of human factors in Web design
  doi: 10.1201/b10855-37
– volume: 59
  start-page: 105
  year: 2004
  end-page: 117
  ident: CR19
  article-title: Psychological research online: Report of Board of Scientific Affairs’ Advisory Group on the Conduct of Research on the Internet
  publication-title: American Psychologist
  doi: 10.1037/0003-066X.59.2.105
– volume: 47
  start-page: 309
  year: 2015
  end-page: 327
  ident: CR29
  article-title: Presentation and response timing accuracy in Adobe Flash and HTML5/JavaScript Web experiments
  publication-title: Behavior Research Methods
  doi: 10.3758/s13428-014-0471-1
– volume: 40
  start-page: 359
  year: 2006
  end-page: 376
  ident: CR4
  article-title: Personality assessment: Does the medium matter? No
  publication-title: Journal of Research in Personality
  doi: 10.1016/j.jrp.2005.01.006
– volume: 11
  start-page: 502
  year: 2000
  end-page: 506
  ident: CR22
  article-title: The integrity of Web-delivered experiments: Can you trust the data?
  publication-title: Psychological Science
  doi: 10.1111/1467-9280.00296
– volume: 33
  start-page: 179
  year: 2001
  end-page: 186
  ident: CR11
  article-title: An inaccurate-timing filter for reaction time measurement by JAVA applets implementing Internet-based experiments
  publication-title: Behavior Research Methods, Instruments, & Computers
  doi: 10.3758/BF03195364
– volume: 41
  start-page: 1
  year: 2009
  end-page: 12
  ident: CR18
  article-title: Timing accuracy of Web experiments: A case study using the WebExp software package
  publication-title: Behavior Research Methods
  doi: 10.3758/BRM.41.1.12
– volume: 57
  start-page: 529
  year: 2006
  end-page: 555
  ident: CR37
  article-title: The Internet as psychological laboratory
  publication-title: Annual Review of Psychology
  doi: 10.1146/annurev.psych.57.102904.190048
– volume: 39
  start-page: 175
  year: 2007
  end-page: 191
  ident: CR12
  article-title: G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences
  publication-title: Behavior Research Methods
  doi: 10.3758/BF03193146
– volume: 43
  start-page: 353
  year: 2011
  end-page: 362
  ident: CR23
  article-title: Response time accuracy in Apple Macintosh computers
  publication-title: Behavior Research Methods
  doi: 10.3758/s13428-011-0069-9
– volume: 30
  start-page: 350
  year: 2012
  end-page: 357
  ident: CR3
  article-title: Assessing the effects of technical variance on the statistical outcomes of web experiments measuring response times
  publication-title: Social Science Computer Review
  doi: 10.1177/0894439311415604
– year: 1988
  ident: CR5
  publication-title: Statistical power analysis for the behavioral sciences
– ident: CR32
– volume: 31
  start-page: 439
  year: 1999
  end-page: 445
  ident: CR16
  article-title: A perception experiment with time-critical graphics animation on the World-Wide Web
  publication-title: Behavior Research Methods, Instruments, & Computers
  doi: 10.3758/BF03200724
– volume: 46
  start-page: 95
  year: 2014
  end-page: 111
  ident: CR36
  article-title: Collecting response times using Amazon Mechanical Turk and Adobe Flash
  publication-title: Behavior Research Methods
  doi: 10.3758/s13428-013-0345-y
– volume: 55
  start-page: 803
  year: 2004
  end-page: 832
  ident: CR2
  article-title: Human research and data collection via the Internet
  publication-title: Annual Review of Psychology
  doi: 10.1146/annurev.psych.55.090902.141601
– volume: 8
  start-page: 1
  issue: e57410
  year: 2013
  ident: 678_CR8
  publication-title: PLoS ONE
  doi: 10.1371/journal.pone.0057410
– volume: 46
  start-page: 95
  year: 2014
  ident: 678_CR36
  publication-title: Behavior Research Methods
  doi: 10.3758/s13428-013-0345-y
– volume: 59
  start-page: 105
  year: 2004
  ident: 678_CR19
  publication-title: American Psychologist
  doi: 10.1037/0003-066x.59.2.105
– volume: 1
  start-page: 71
  year: 2006
  ident: 678_CR24
  publication-title: International Journal of Internet Science
– volume: 39
  start-page: 365
  year: 2007
  ident: 678_CR28
  publication-title: Behavior Research Methods
  doi: 10.3758/bf03193004
– volume: 9
  start-page: 126
  year: 2002
  ident: 678_CR6
  publication-title: Psychonomic Bulletin & Review
  doi: 10.3758/bf03196267
– volume: 41
  start-page: 1
  year: 2009
  ident: 678_CR18
  publication-title: Behavior Research Methods
  doi: 10.3758/BRM.41.1.12
– volume: 41
  start-page: 1183
  year: 2009
  ident: 678_CR17
  publication-title: Behavior Research Methods
  doi: 10.3758/BRM.41.4.1183
– volume-title: E-Prime: User’s guide
  year: 2002
  ident: 678_CR35
– year: 2015
  ident: 678_CR10
  publication-title: Behavior Research Methods
  doi: 10.3758/s13428-015-0567-2
– volume: 30
  start-page: 350
  year: 2012
  ident: 678_CR3
  publication-title: Social Science Computer Review
  doi: 10.1177/0894439311415604
– volume: 9
  start-page: 487
  year: 1970
  ident: 678_CR34
  publication-title: Journal of Verbal Learning and Verbal Behavior
  doi: 10.1016/S0022-5371(70)80091-3
– volume: 86
  start-page: 446
  year: 1979
  ident: 678_CR27
  publication-title: Psychological Bulletin
  doi: 10.1037/0033-2909.86.3.446
– volume: 43
  start-page: 548
  year: 2011
  ident: 678_CR20
  publication-title: Behavior Research Methods
  doi: 10.3758/s13428-011-0066-z
– volume: 33
  start-page: 179
  year: 2001
  ident: 678_CR11
  publication-title: Behavior Research Methods, Instruments, & Computers
  doi: 10.3758/BF03195364
– volume: 41
  start-page: 598
  year: 2009
  ident: 678_CR26
  publication-title: Behavior Research Methods
  doi: 10.3758/BRM.41.3.598
– ident: 678_CR32
– volume: 35
  start-page: 276
  year: 2003
  ident: 678_CR25
  publication-title: Behavior Research Methods, Instruments, & Computers
  doi: 10.3758/bf03202553
– volume: 22
  start-page: 448
  year: 2006
  ident: 678_CR21
  publication-title: Computers in Human Behavior
  doi: 10.1016/j.chb.2004.09.010
– volume: 20
  start-page: 241
  year: 2002
  ident: 678_CR30
  publication-title: Social Science Computer Review
  doi: 10.1177/08939302020003002
– start-page: 563
  volume-title: The handbook of human factors in Web design
  year: 2011
  ident: 678_CR31
  doi: 10.1201/b10855-37
– volume-title: Statistical power analysis for the behavioral sciences
  year: 1988
  ident: 678_CR5
– volume: 40
  start-page: 359
  year: 2006
  ident: 678_CR4
  publication-title: Journal of Research in Personality
  doi: 10.1016/j.jrp.2005.01.006
– volume: 39
  start-page: 175
  year: 2007
  ident: 678_CR12
  publication-title: Behavior Research Methods
  doi: 10.3758/BF03193146
– volume: 43
  start-page: 353
  year: 2011
  ident: 678_CR23
  publication-title: Behavior Research Methods
  doi: 10.3758/s13428-011-0069-9
– volume: 22
  start-page: 24
  year: 1983
  ident: 678_CR14
  publication-title: Journal of Verbal Learning and Verbal Behavior
  doi: 10.1016/S0022-5371(83)80004-8
– volume: 36
  start-page: 1339
  year: 2010
  ident: 678_CR1
  publication-title: Journal of Experimental Psychology: Learning, Memory, and Cognition
  doi: 10.1037/a0019999
– volume: 57
  start-page: 529
  year: 2006
  ident: 678_CR37
  publication-title: Annual Review of Psychology
  doi: 10.1146/annurev.psych.57.102904.190048
– volume: 34
  start-page: 177
  year: 2002
  ident: 678_CR7
  publication-title: Behavior Research Methods, Instruments, & Computers
  doi: 10.3758/BF03195440
– volume: 19
  start-page: 847
  year: 2012
  ident: 678_CR13
  publication-title: Psychonomic Bulletin & Review
  doi: 10.3758/s13423-012-0296-9
– volume: 55
  start-page: 803
  year: 2004
  ident: 678_CR2
  publication-title: Annual Review of Psychology
  doi: 10.1146/annurev.psych.55.090902.141601
– volume: 47
  start-page: 1
  year: 2015
  ident: 678_CR9
  publication-title: Behavior Research Methods
  doi: 10.3758/s13428-014-0458-y
– volume: 11
  start-page: 502
  year: 2000
  ident: 678_CR22
  publication-title: Psychological Science
  doi: 10.1111/1467-9280.00296
– start-page: 193
  volume-title: Advanced methods for conducting online behavioral research
  year: 2010
  ident: 678_CR33
  doi: 10.1037/12076-013
– volume: 47
  start-page: 309
  year: 2015
  ident: 678_CR29
  publication-title: Behavior Research Methods
  doi: 10.3758/s13428-014-0471-1
– volume: 59
  start-page: 93
  year: 2004
  ident: 678_CR15
  publication-title: American Psychologist
  doi: 10.1037/0003-066X.59.2.93
– volume: 31
  start-page: 439
  year: 1999
  ident: 678_CR16
  publication-title: Behavior Research Methods, Instruments, & Computers
  doi: 10.3758/bf03200724
SSID ssj0038214
Score 2.5437827
Snippet Although Web-based research is now commonplace, it continues to spur skepticism from reviewers and editors, especially whenever reaction times are of primary...
SourceID proquest
pubmed
crossref
springer
SourceType Aggregation Database
Index Database
Enrichment Source
Publisher
StartPage 1718
SubjectTerms Behavioral Research - methods
Behavioral Science and Psychology
Cognitive Psychology
Data Collection
Humans
Internet
Psychology
Reaction Time - physiology
Software
Young Adult
Title Reaction time effects in lab- versus Web-based research: Experimental evidence
URI https://link.springer.com/article/10.3758/s13428-015-0678-9
https://www.ncbi.nlm.nih.gov/pubmed/26542972
https://www.proquest.com/docview/1826632579
Volume 48
WOSCitedRecordID wos000392719000040&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVAVX
  databaseName: SpringerLink Contemporary
  customDbUrl:
  eissn: 1554-3528
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0038214
  issn: 1554-3528
  databaseCode: RSV
  dateStart: 20050201
  isFulltext: true
  titleUrlDefault: https://link.springer.com/search?facet-content-type=%22Journal%22
  providerName: Springer Nature
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1LSwMxEB5s9dCL70d9lAielOBuNrvJehOxeCpSX72FZJNgQbbSbQX_vUl3tyhVQe-TzTLvYSbzAZzEPAm0NAYbKiWmkZVYJanCmipj_T45bUuwCdbr8cEgva3ecRf1tHvdkpx5al9XuqT2vAgj6rcphzH2HhanDVh20Y57a-zfPdbuN-IkpGX78vtjXwPQQla50BGdBZru2r9-cR1Wq7wSXZaKsAFLJt-E1ty9vW9Br2_KRwzI48mjapADDXPkFAEjP54xLdCTUdhHNo2qNUDPF-j6EwoAMhUM6TY8dK_vr25whaaAs5gEE8yVy9QYt1xaEshMaSa5IilLbcxdHCNJxlOrg4jqgCpPYww3RDOuDOOB1tEONPNRbvYAMapDV8hZQ1LnaImULisgihIbh4rHVLYhqFkssmrVuEe8eBGu5PCcEiWnhOOU8JwSaRtO50deyz0bvxEf13ITzhp8i0PmZjQthK-Wksi5IUezWwp0_jkyw-ZipA1ntfREZbDFz3ft_4n6AFouo0rKeZdDaE7GU3MEK9nbZFiMO9BgA96ZqesHmqjkfw
linkProvider Springer Nature
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1ZS8QwEB68QF-8j_WM4JMSbNO0TX0TcVlRi3juW0iaBBekit0V_PcmPRZlVdD3SVMmk29mmMl8AHshizwltMaaCoFpYASWUSKxolIbN09OmYpsIk5T1u0mV_U77qLpdm9KkiVSu7zSBrWHhR9QN03ZD7FDWJyMwyS1Dsv18V3f3DfwGzDi06p8-f2yrw5oJKocqYiWjqY9969fnIfZOq5Ex5UhLMCYzhdhZghv70uQXuvqEQNyfPKobuRAvRxZQ8DItWcMCvSgJXaeTaF6DNDjETr9xAKAdE1Dugx37dPbkw6u2RRwFhKvj5m0kVrMDBOGeCKTKhZMkiROTMisHyNRxhKjvIAqj0onozXTRMVM6ph5SgUrMJE_53oNUEyVbxM5o0ligZYIYaMCIikxoS9ZSEULvEbFPKtHjTvGiyduUw6nKV5piltNcacpnrRgf7jkpZqz8ZvwbnNu3N4GV-IQuX4eFNxlS1FgYcjKrFYHOvwcKbm5YtKCg-b0eH1hi5_3Wv-T9A5Md24vL_jFWXq-ATM2uoqq3pdNmOi_DvQWTGVv_V7xul0a7Qf3S-Z7
linkToPdf http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1ZS8QwEB50FfHF-1jPCD4pYds0bVPfRF0UZVm830LSJLggVeyu4L832bSL4gHi-6Qpcw8zmQ9gN2ZJoITWWFMhMI2MwDLJJFZUauP2ySnjwSbSTofd32fdCue0rKfd65akf9PgtjQV_dazMs7EI5vgtsowom6zchhj521xNg4T1GEGuXL96rZ2xREjIfWtzO-PfQ5GXzLML93RYdBpz_77d-dgpso30aFXkHkY08UCTI_c3tsidC61f9yAHM48qgY8UK9AVkEwcmMbgxLdaYldxFOoWg_0cIBOPqADIF3Bky7BTfvk-ugUVygLOI9J0MdM2gwuZYYJQwKRS5UKJkmWZiZmNr6RJGeZUUFEVUClo9GaaaJSJnXKAqWiZWgUT4VeBZRSFdoCz2iSWQdMhLDZApGUmDiULKaiCUHNbp5XK8gdEsYjt6WI4xT3nOKWU9xximdN2Bsdefb7N34j3qllyK2VuNaHKPTToOSuikoi654szYoX7uhzZIjZlZIm7NeS5JUhlz_ftfYn6m2Y6h63-cVZ53wdpm3SlfiRmA1o9F8GehMm89d-r3zZGurvOwXv718
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Reaction+time+effects+in+lab-+versus+Web-based+research%3A+Experimental+evidence&rft.jtitle=Behavior+research+methods&rft.au=Hilbig%2C+Benjamin+E.&rft.date=2016-12-01&rft.pub=Springer+US&rft.eissn=1554-3528&rft.volume=48&rft.issue=4&rft.spage=1718&rft.epage=1724&rft_id=info:doi/10.3758%2Fs13428-015-0678-9&rft.externalDocID=10_3758_s13428_015_0678_9
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1554-3528&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1554-3528&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1554-3528&client=summon