Running Experiments on Amazon Mechanical Turk

Although Mechanical Turk has recently become popular among social scientists as a source of experimental data, doubts may linger about the quality of data provided by subjects recruited from online labor markets. We address these potential concerns by presenting new demographic data about the Mechan...

Full description

Saved in:
Bibliographic Details
Published in:Judgment and Decision Making Vol. 5; no. 5; pp. 411 - 419
Main Authors: Paolacci, Gabriele, Chandler, Jesse, Ipeirotis, Panagiotis G.
Format: Journal Article
Language:English
Published: Tallahassee Society for Judgment and Decision Making 01.08.2010
Cambridge University Press
Series:Judgment and Decision Making
Subjects:
Decision making
Demographics
experimentation
Experiments
Income distribution
Internet
Labor market
online research
Population
Wages & salaries
Workers
online research
experimentation
ISSN:1930-2975, 1930-2975
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Although Mechanical Turk has recently become popular among social scientists as a source of experimental data, doubts may linger about the quality of data provided by subjects recruited from online labor markets. We address these potential concerns by presenting new demographic data about the Mechanical Turk subject population, reviewing the strengths of Mechanical Turk relative to other online and offline methods of recruiting subjects, and comparing the magnitude of effects obtained using Mechanical Turk and traditional subject pools. We further discuss some additional benefits such as the possibility of longitudinal, cross cultural and prescreening designs, and offer some advice on how to best manage a common subject pool.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1930-2975
1930-2975
DOI:10.1017/s1930297500002205