An approach for selecting and using a method of inter-coder reliability in information management research

•Qualitative codes can be trusted only after ensuring their reliability.•Checking the inter-coder reliability prevents extra time, effort and financial costs.•Percent agreement is the least and Krippendorff's alpha is the most flexible method.•The right method is that matches the characteristic...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:International journal of information management Ročník 54; s. 102154
Hlavní autori: Nili, Alireza, Tate, Mary, Barros, Alistair, Johnstone, David
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Kidlington Elsevier Ltd 01.10.2020
Elsevier Science Ltd
Predmet:
Coders
Content analysis
Credibility
Discrepancies
Errors
Evaluation
Information
Information management
Inter-coder reliability
Management
Measurement
Qualitative analysis
Qualitative research
Reliability
Reliability analysis
Researchers
Test validity and reliability
Management
Information
Inter-coder reliability
Content analysis
ISSN:0268-4012, 1873-4707
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:•Qualitative codes can be trusted only after ensuring their reliability.•Checking the inter-coder reliability prevents extra time, effort and financial costs.•Percent agreement is the least and Krippendorff's alpha is the most flexible method.•The right method is that matches the characteristics of a specific project.•The process of selecting and using the right method includes five major steps. Qualitative researchers in information management research often need to evaluate inter-coder reliability (ICR) to test the trustworthiness of their content analysis. A suitable method of evaluating ICR enables researchers to rigorously assess the degree of agreement among two or more independent qualitative coders. This allows researchers to identify mistakes in the content analysis before the codes are used in developing and testing a theory or a measurement model and avoid any associated time, effort and financial cost. Different methods have been proposed, but little guidance is available on which approach to evaluating ICR should be used. In this paper, we review and compare leading ICR methods that are suitable for qualitative information management research. We propose an approach for selecting and using an ICR method, supported by an illustrative example. The five steps in our proposed approach include: selecting an ICR method based on its characteristics and requirements of a project; developing a coding scheme; selecting and training independent coders; calculating the ICR coefficient and resolving discrepancies; and reporting the process of evaluating ICR and its results.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0268-4012
1873-4707
DOI:10.1016/j.ijinfomgt.2020.102154