Intercoder Reliability for Use in Qualitative Research and Evaluation

Intercoder reliability can increase trustworthiness, accuracy, rigor, collaboration, and power sharing in qualitative research. Though not every qualitative design can utilize intercoder reliability, this article highlights how positivist qualitative research, community-based participatory research,...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Measurement and evaluation in counseling and development Jg. 57; H. 2; S. 136 - 146
Hauptverfasser: Coleman, Monica L., Ragan, Moira, Dari, Tahani
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Alexandria Taylor & Francis Ltd 02.04.2024
Schlagworte:
Collaboration
Community based action research
Community research
Cooperative learning
Counseling
Counselor Training
Counselors
Credibility
Education
Evaluation
Evaluation research
Evaluators
Kappa statistic
Participatory research
Power-sharing
Qualitative research
Reliability
Teams
Trainees
ISSN:0748-1756, 1947-6302
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Abstract Intercoder reliability can increase trustworthiness, accuracy, rigor, collaboration, and power sharing in qualitative research. Though not every qualitative design can utilize intercoder reliability, this article highlights how positivist qualitative research, community-based participatory research, and participatory evaluation all strengthen when intercoder reliability is sought among coding teams. Basic calculations for intercoder reliability can be completed manually or with software, making it an accessible metric to inform counseling research and evaluation, benefitting counselor education and practice. Interpreting metrics like percent agreement or kappa statistic largely depends on data context and potential impact of the study, which warrants researchers and evaluators to assess and understand the consequences of their work. Using ICR with clinical data such as interviews and other qualitative sources can be utilized to teach trainees the method in counseling research and evaluation and help make visible the important contributions of qualitative research using collaborative teams in counselor education.
AbstractList Intercoder reliability can increase trustworthiness, accuracy, rigor, collaboration, and power sharing in qualitative research. Though not every qualitative design can utilize intercoder reliability, this article highlights how positivist qualitative research, community-based participatory research, and participatory evaluation all strengthen when intercoder reliability is sought among coding teams. Basic calculations for intercoder reliability can be completed manually or with software, making it an accessible metric to inform counseling research and evaluation, benefitting counselor education and practice. Interpreting metrics like percent agreement or kappa statistic largely depends on data context and potential impact of the study, which warrants researchers and evaluators to assess and understand the consequences of their work. Using ICR with clinical data such as interviews and other qualitative sources can be utilized to teach trainees the method in counseling research and evaluation and help make visible the important contributions of qualitative research using collaborative teams in counselor education.
Author Ragan, Moira
Coleman, Monica L.
Dari, Tahani
Author_xml – sequence: 1
  givenname: Monica L.
  orcidid: 0000-0002-0513-0221
  surname: Coleman
  fullname: Coleman, Monica L.
  organization: University of Mississippi, Oxford, MS, USA
– sequence: 2
  givenname: Moira
  orcidid: 0009-0003-1404-6675
  surname: Ragan
  fullname: Ragan, Moira
  organization: Ripple Effect, Rockville, MD, USA
– sequence: 3
  givenname: Tahani
  orcidid: 0000-0002-5873-5435
  surname: Dari
  fullname: Dari, Tahani
  organization: University of Detroit Mercy, Detroit, MI, USA
BookMark eNqFkN1KAzEQhYNUsK0-grDg9dZJ9ie7eCWl1UJBFHsdsvnBlDWpyW6hb-Oz-GRmbb3xxothYM75ZpgzQSPrrELoGsMMQwW3QPMK06KcESD5jGSQUVycoTGuc5qWGZARGg-edDBdoEkIWwDAFcAYLVe2U144qXzyolrDG9Oa7pBo55NNUImxyXPP44h3Zq-iJSjuxVvCrfz6XOx520fB2Ut0rnkb1NWpT9FmuXidP6brp4fV_H6dClLhLs0VF1TLPGuIkHWudEGpKCpdSYm5brTWCrjMOZGyUqWopSqbGmNSRzVyTTZFN8e9O-8-ehU6tnW9t_EkG96mRQaxpuju6BLeheCVZuLnAWc7z03LMLAhOPYbHBuCY6fgIl38oXfevHN_-If7BqKWdOw
CitedBy_id crossref_primary_10_3390_logistics9010014
crossref_primary_10_1002_jmcd_12309
crossref_primary_10_1007_s11301_025_00499_4
crossref_primary_10_1016_j_brat_2024_104665
crossref_primary_10_1007_s10826_025_03014_1
crossref_primary_10_1080_07481756_2024_2333692
crossref_primary_10_1080_03075079_2024_2405559
crossref_primary_10_1007_s11191_025_00677_6
crossref_primary_10_1080_07481756_2025_2489980
Cites_doi 10.1177/1609406919899220
10.1080/07481756.2017.1388680
10.1007/s11135-017-0607-3
10.1177/1098214013478142
10.1027/1614-2241/a000086
10.15241/td.9.1.1
10.1080/02635143.2021.1993179
10.1016/0147-1767(85)90062-8
10.2307/2529310
10.1177/1468794115577012
10.1080/03069885.2014.936823
10.1111/j.1468-2958.2004.tb00738.x
10.1177/001316446002000104
10.1080/07481756.2022.2052321
10.1177/1525822X0001200301
10.11613/BM.2012.031
10.4135/9781526430212
10.1080/21501378.2022.2025771
ContentType Journal Article
Copyright 2024 Association for Assessment and Research in Counseling (AARC)
Copyright_xml – notice: 2024 Association for Assessment and Research in Counseling (AARC)
DBID AAYXX
CITATION
7QJ
AHOVV
DOI 10.1080/07481756.2024.2303715
DatabaseName CrossRef
Applied Social Sciences Index & Abstracts (ASSIA)
Education Research Index
DatabaseTitle CrossRef
Applied Social Sciences Index and Abstracts (ASSIA)
DatabaseTitleList Applied Social Sciences Index and Abstracts (ASSIA)
DeliveryMethod fulltext_linktorsrc
Discipline Education
Psychology
EISSN 1947-6302
EndPage 146
ExternalDocumentID 10_1080_07481756_2024_2303715
GroupedDBID -W8
-~X
.GO
01A
0BK
0R~
18M
29M
4.4
8R4
8R5
AAGDL
AAHIA
AAHSB
AAJPV
AAMFJ
AAMIU
AAPUL
AAWTL
AAYXX
AAZMC
ABAWP
ABDBF
ABDPE
ABIVO
ABJNI
ABLIJ
ABPPZ
ABRYG
ABUFD
ABXUL
ABXYU
ACDXX
ACGFO
ACGFS
ACHQT
ACNCT
ACTIO
ACUHS
ADAHI
ADEBD
ADIUE
ADKVQ
AECIN
AEFOU
AEISY
AEUHG
AEYOC
AEZRU
AFHDM
AFRVT
AGDLA
AGRBW
AHDZW
AIJEM
AIYEW
AKBVH
ALMA_UNASSIGNED_HOLDINGS
ALQZU
ARTOV
AWYRJ
AZFZN
B0M
BLEHA
BMOTO
BOHLJ
CCCUG
CITATION
DGFLZ
DO4
DU5
EAP
EAS
EBS
ECR
EDJ
EMF
EMH
EMK
EPL
EPS
ESX
GROUPED_SAGE_PREMIER_JOURNAL_COLLECTION
H13
HF~
HZ~
IAO
IEA
IOF
IPNFZ
IPY
ITC
J8X
KYCEM
L7B
LJTGL
M0P
M4Z
N9A
O9-
P.B
P2P
Q2X
RIG
RNANH
ROSJB
RPD
RSYQP
RWL
RXW
S01
SFC
TAE
TASJS
TBQAZ
TDBHL
TFH
TFL
TFW
TN5
TNJ
TNTFI
TRJHH
TUROJ
TUS
WH7
YNT
YQT
~8M
7QJ
AHOVV
AQTUD
ID FETCH-LOGICAL-c281t-4eac7fd43b2cd94ef577c58f8dd1afbfffe0ad4a2dd8e6c9de6b911291af7fdb3
ISICitedReferencesCount 13
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001174175200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 0748-1756
IngestDate Sun Nov 16 12:10:51 EST 2025
Sat Nov 29 02:59:33 EST 2025
Tue Nov 18 21:23:04 EST 2025
IsPeerReviewed true
IsScholarly true
Issue 2
Language English
LinkModel OpenURL
MergedId FETCHMERGED-LOGICAL-c281t-4eac7fd43b2cd94ef577c58f8dd1afbfffe0ad4a2dd8e6c9de6b911291af7fdb3
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0000-0002-0513-0221
0009-0003-1404-6675
0000-0002-5873-5435
PQID 3037753075
PQPubID 34423
PageCount 11
ParticipantIDs proquest_journals_3037753075
crossref_citationtrail_10_1080_07481756_2024_2303715
crossref_primary_10_1080_07481756_2024_2303715
PublicationCentury 2000
PublicationDate 2024-04-02
PublicationDateYYYYMMDD 2024-04-02
PublicationDate_xml – month: 04
  year: 2024
  text: 2024-04-02
  day: 02
PublicationDecade 2020
PublicationPlace Alexandria
PublicationPlace_xml – name: Alexandria
PublicationTitle Measurement and evaluation in counseling and development
PublicationYear 2024
Publisher Taylor & Francis Ltd
Publisher_xml – name: Taylor & Francis Ltd
References IBM Corp (e_1_3_3_13_1) 2021
e_1_3_3_18_1
e_1_3_3_17_1
Creswell J. W. (e_1_3_3_9_1) 2018
e_1_3_3_19_1
e_1_3_3_14_1
e_1_3_3_16_1
e_1_3_3_15_1
e_1_3_3_10_1
e_1_3_3_11_1
Miles M. B. (e_1_3_3_23_1) 2020
Freelon D. (e_1_3_3_12_1) 2013; 8
e_1_3_3_7_1
e_1_3_3_6_1
e_1_3_3_8_1
e_1_3_3_25_1
e_1_3_3_24_1
e_1_3_3_26_1
e_1_3_3_3_1
e_1_3_3_2_1
e_1_3_3_20_1
Marshall C. (e_1_3_3_21_1) 1999
e_1_3_3_5_1
e_1_3_3_4_1
e_1_3_3_22_1
References_xml – ident: e_1_3_3_2_1
– ident: e_1_3_3_24_1
  doi: 10.1177/1609406919899220
– ident: e_1_3_3_3_1
  doi: 10.1080/07481756.2017.1388680
– ident: e_1_3_3_4_1
  doi: 10.1007/s11135-017-0607-3
– ident: e_1_3_3_6_1
  doi: 10.1177/1098214013478142
– ident: e_1_3_3_11_1
  doi: 10.1027/1614-2241/a000086
– ident: e_1_3_3_10_1
  doi: 10.15241/td.9.1.1
– ident: e_1_3_3_5_1
  doi: 10.1080/02635143.2021.1993179
– volume-title: Designing qualitative research
  year: 1999
  ident: e_1_3_3_21_1
– ident: e_1_3_3_18_1
  doi: 10.1016/0147-1767(85)90062-8
– ident: e_1_3_3_16_1
  doi: 10.2307/2529310
– volume-title: IBM SPSS Statistics for Windows, Version 28.0
  year: 2021
  ident: e_1_3_3_13_1
– ident: e_1_3_3_20_1
  doi: 10.1177/1468794115577012
– ident: e_1_3_3_26_1
  doi: 10.1080/03069885.2014.936823
– ident: e_1_3_3_14_1
  doi: 10.1111/j.1468-2958.2004.tb00738.x
– ident: e_1_3_3_19_1
– ident: e_1_3_3_7_1
  doi: 10.1177/001316446002000104
– ident: e_1_3_3_17_1
  doi: 10.1080/07481756.2022.2052321
– ident: e_1_3_3_15_1
  doi: 10.1177/1525822X0001200301
– volume-title: Qualitative data analysis: A methods sourcebook
  year: 2020
  ident: e_1_3_3_23_1
– volume-title: Research design: Qualitative, quantitative, and mixed methods approaches
  year: 2018
  ident: e_1_3_3_9_1
– ident: e_1_3_3_22_1
  doi: 10.11613/BM.2012.031
– ident: e_1_3_3_25_1
  doi: 10.4135/9781526430212
– volume: 8
  start-page: 10
  issue: 1
  year: 2013
  ident: e_1_3_3_12_1
  article-title: ReCal OIR: Ordinal, interval, and ratio intercoder reliability as a web service
  publication-title: International Journal of Internet Science,
– ident: e_1_3_3_8_1
  doi: 10.1080/21501378.2022.2025771
SSID ssj0001800
Score 2.4090044
Snippet Intercoder reliability can increase trustworthiness, accuracy, rigor, collaboration, and power sharing in qualitative research. Though not every qualitative...
SourceID proquest
crossref
SourceType Aggregation Database
Enrichment Source
Index Database
StartPage 136
SubjectTerms Collaboration
Community based action research
Community research
Cooperative learning
Counseling
Counselor Training
Counselors
Credibility
Education
Evaluation
Evaluation research
Evaluators
Kappa statistic
Participatory research
Power-sharing
Qualitative research
Reliability
Teams
Trainees
Title Intercoder Reliability for Use in Qualitative Research and Evaluation
URI https://www.proquest.com/docview/3037753075
Volume 57
WOSCitedRecordID wos001174175200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVAWR
  databaseName: Taylor & Francis Online
  customDbUrl:
  eissn: 1947-6302
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0001800
  issn: 0748-1756
  databaseCode: TFW
  dateStart: 19840401
  isFulltext: true
  titleUrlDefault: https://www.tandfonline.com
  providerName: Taylor & Francis
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV3LTtwwFLWmlAWbqqVFpUDlRVmhjPK0nSWijFgAYjGo7CLHDxhpFNDMgOjfVHwKX8Z1_EimVC0suolG8cTJ5J7xvb4-9xihbyqraw1uIlKZiiPwECJi8N2o0KWMC0mlsIXCx_T0lF1clGeDwYOvhbmb0qZh9_flzX81NZwDY5vS2VeYO3QKJ-AzGB2OYHY4vsjwbY7PVKrPWr6x1eG2tMzzeSsRYnUzrOK3Z96ZRYTdg3R3Pz4M-t_9wPWkyyW26w2dSvhey2QPle2mUXZMpLDGcT1VLtlqhhHB946H3RrTpW-ZzIKb-M5tEfyYX_Fm0s9OpJbU0k9Y0hxmqbRwctd2kC1zGpEsXhqFrUy1Q1vaG1KTjPS8s0tYPhv4HVMS7mZuNjSPYjjuGbXFostC2785wEBLTLxequumMt1Urps36G1Ki9LQBsejH8HbJ8yWOfnf6avEjH77n55mOf5Zdv9tTDN-j965yQjetyD6gAaqWTf7eDvOzzpaC_7x50c06pCFe8jCgCwMyMKTBveQhT2yMCDi8VeHqk_ofHQ4PjiK3DYckUhZsohy8M1UyzyrUyHLXOmCUlEwzaRMuK611irmMueplEwRUUpF6tKE8dAK19XZBlpprhv1GWGSkziDELimMBEuiOaMCJIoUcQJJ6ommyj3L6cSTqPebJUyrf5qmk00DJfdWJGWf12w7d985f6688o0weQdYugvr-1vC6114N9GK4vZrdpBq-JuMZnPvraAeQJ2FZEb
linkProvider Taylor & Francis
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Intercoder+Reliability+for+Use+in+Qualitative+Research+and%C2%A0Evaluation&rft.jtitle=Measurement+and+evaluation+in+counseling+and+development&rft.au=Coleman%2C+Monica+L.&rft.au=Ragan%2C+Moira&rft.au=Dari%2C+Tahani&rft.date=2024-04-02&rft.issn=0748-1756&rft.eissn=1947-6302&rft.volume=57&rft.issue=2&rft.spage=136&rft.epage=146&rft_id=info:doi/10.1080%2F07481756.2024.2303715&rft.externalDBID=n%2Fa&rft.externalDocID=10_1080_07481756_2024_2303715
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0748-1756&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0748-1756&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0748-1756&client=summon