Developing Shared Ways of Seeing Data: The Perils and Possibilities of Achieving Intercoder Agreement

All research, e.g., qualitative or quantitative, is concerned with the extent to which analyses can adequately describe the phenomena it seeks to describe. In qualitative research, we use internal validity checks like intercoder agreement to measure the extent to which independent researchers observ...

Full description

Saved in:
Bibliographic Details
Published in:International journal of qualitative methods Vol. 22
Main Authors: Clarke, Sherice N., Sushil, S, Dennis, Katherine, Lee, Ung-Sang, Gomoll, Andrea, Gates, Zaynab
Format: Journal Article
Language:English
Published: Los Angeles, CA SAGE Publications 01.02.2023
Sage Publications Ltd
SAGE Publishing
Subjects:
Agreements
Alignment
Credibility
Data
Data analysis
Inference
Internal validity
Qualitative research
Teams
qualitative data analysis
intercoder agreement
trustworthiness
intercoder reliability
team-based research
qualitative coding
ISSN:1609-4069, 1609-4069
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:All research, e.g., qualitative or quantitative, is concerned with the extent to which analyses can adequately describe the phenomena it seeks to describe. In qualitative research, we use internal validity checks like intercoder agreement to measure the extent to which independent researchers observe the same phenomena in data. Researchers report indices of agreement to serve as evidence of consistency and dependability of interpretations, and we do so to make claims about the trustworthiness of our research accounts. However, few studies report the methods of how multiple analysts developed alignment in their interpretation of data, a process that undergirds accounts of consistency, dependability, and trustworthiness. In this article, we review the issues and options around achieving intercoder agreement. Drawing on our experience from a longitudinal, team-based research project that required rapid cycles of qualitative data analysis, we reflect on the challenges we had achieving high intercoder agreement (which refer to as the perils). It was through these challenges that we developed a method that helps to foster shared ways of seeing data, and thus alignment in our interpretations of phenomena in data. In this article, we present this method as a tool for dyadic and team-based qualitative data analysis to facilitate reliable and consistent high-inference interpretations of data with multiple analysts.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1609-4069
1609-4069
DOI:10.1177/16094069231160973