Coding Qualitative Data at Scale: Guidance for Large Coder Teams Based on 18 Studies

We outline a process for using large coder teams (10 + coders) to code large-scale qualitative data sets. The process reflects experience recruiting and managing large teams of novice and trainee coders for 18 projects in the last decade, each engaging a coding team of 12 (minimum) to 54 (maximum) c...

Full description

Saved in:
Bibliographic Details
Published in:International journal of qualitative methods Vol. 21
Main Authors: Beresford, Melissa, Wutich, Amber, du Bray, Margaret V., Ruth, Alissa, Stotts, Rhian, SturtzSreetharan, Cindi, Brewis, Alexandra
Format: Journal Article
Language:English
Published: Los Angeles, CA SAGE Publications 11.01.2022
Sage Publications Ltd
SAGE Publishing
Subjects:
Compensation
Data quality
Hiring
Morale
Nonprofit organizations
Qualitative research
Recruitment
Reliability
Teams
qualitative data analysis
intercoder agreement
text analysis
collaborative research
qualitative coding
team-based coding
intercoder reliability
ISSN:1609-4069, 1609-4069
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We outline a process for using large coder teams (10 + coders) to code large-scale qualitative data sets. The process reflects experience recruiting and managing large teams of novice and trainee coders for 18 projects in the last decade, each engaging a coding team of 12 (minimum) to 54 (maximum) coders. We identify four unique challenges to large coder teams that are not presently discussed in the methodological literature: (1) recruiting and training coders, (2) providing coder compensation and incentives, (3) maintaining data quality and ensuring coding reliability at scale, and (4) building team cohesion and morale. For each challenge, we provide associated guidance. We conclude with a discussion of advantages and disadvantages of large coder teams for qualitative research and provide notes of caution for anyone considering hiring and/or managing large coder teams for research (whether in academia, government and non-profit sectors, or industry).
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1609-4069
1609-4069
DOI:10.1177/16094069221075860